Magic Quadrant for Application Development Life Cycle Management

19 November 2013 ID:G00249074
Analyst(s): Thomas E. Murphy, Jim Duggan, Nathan Wilson

VIEW SUMMARY

Time-to-market pressure is driving the application development life cycle management market to evolve and expand. This research evaluates leading ADLM providers to help application development and IT managers select appropriate technology partners that drive quality and productivity.

Market Definition/Description

The application development life cycle management (ADLM) tool market is focused on the planning and governance activities of the software development life cycle (SDLC). ADLM products are focused on the "development" portion of an application's life, with the term ADLM evolving from the term application life cycle management (ALM).

Key elements of an ADLM solution include:

  • Software requirements definition and management
  • Software change and configuration management
  • Software project planning, with a current focus on agile planning
  • Work item management
  • Quality management, including defect management

In addition, other key capabilities include:

  • Reporting
  • Workflow
  • Integration to version management
  • Support for wikis and collaboration
  • Strong facilities for integration to other ADLM tools

This Magic Quadrant represents a snapshot of the ADLM market at a particular point in time. Gartner advises readers not to compare the placement of vendors from prior years. The market is changing — vendor acquisitions, partnerships, solution development and alternative delivery are evidence of these changes — and the criteria for selecting and ranking vendors continue to evolve. Our assessments take into account the vendors' current product offerings and overall strategies, as well as their future initiatives and product road maps. We also factor in how well vendors are driving market changes or adapting to changing market requirements. This Magic Quadrant will help CIOs and business and IT leaders who are developing their ADLM strategies to assess whether they have the right products and enterprise platforms to support them.

Gartner advises organizations against simply selecting vendors that appear in the Leaders quadrant. All selections should be buyer-specific, and vendors from the Challengers, Niche Players and/or Visionaries quadrants may be better matches for your business goals and solution requirements.

We have shifted terminology in this Magic Quadrant from "application life cycle management" to "application development life cycle management," because we feel it is a more accurate description of the position of these products, which govern the work of an application development project, not the life of the application.

Magic Quadrant

Figure 1. Magic Quadrant for Application Development Life Cycle Management
Figure 1.Magic Quadrant for Application Development Life Cycle Management

Source: Gartner (November 2013)

Vendor Strengths and Cautions

Atlassian

We have rated Atlassian as a Leader that has strong market momentum and acceptance. The vendor's Jira product appears as one of the most frequently deployed tools in Gartner surveys and client interactions. It has broad adoption for issue management, with strong growth in project tracking. Atlassian has a solid investment in agile and support of collaborative development. The vendor has breadth across the life cycle, from planning to continuous integration. The products support a market-leading number of languages and have proven scale in deployments of more than 20,000 users.

The product line takes a concept-to-launch perspective, with a high degree of focus on collaboration support, and ties everything back to the fundamental focus around tasks. Atlassian invests heavily in integration features and has support from other integration products, such as Tasktop.

The vendor offers its solutions on-premises and via SaaS, and it expects most organizations to use a mix of both. However, some extensions are only supported in on-premises installations. Users like the flexibility of choosing feature sets, while still having a high degree of integration, flexibility and a high-value pricing model. Customer support is strong but, unlike other agile vendors, there is not a strong training group and the lack of a direct sales force challenges some of the largest users.

Strengths
  • A large customer base from the vendor's Jira issue management solution drives a strong third-party market.
  • Flexible packaging and licensing options enable users to select needed functions.
  • It is easy to customize workflows and other aspects of the products, and collaboration via HipChat and Confluence extends market reach beyond the core application development team.
  • Solid support for Git and continuous integration positions Atlassian well for organizations adopting DevOps practices.
Cautions
  • Large accounts enjoy low prices, but also miss traditional account executives, and find that contract terms can be inflexible and poorly defined at times.
  • While there are a lot of integrations, import and export functionality is poor.
  • Although the user interface is improving (with a solid improvement in the 6.0 release), it has lagged behind others in the market. Mobile support is still emerging.

Borland

We rate Borland (a Micro Focus company) as a Niche Player. This vendor appeals primarily to its current customer base with a solid set of tools covering the core functions of ADLM, and a strong heritage in developer productivity.

The product line is well-integrated across the life cycle, with a focus on change management and traceability. Borland also provides a broad set of integrations to other widely adopted life cycle products. While StarTeam has strong agile planning capabilities, product visibility in this space has been poor. Core functions in requirements, quality and change management are mature and flexible; Borland Connect, a new integration framework that currently is only used in StarTeam, will eventually be used across all the ADLM tools.

The need for the vendor to spend time on integrating and modernizing its product line has resulted in a stronger set of tools. However, this requirement has also limited the ability to address the pace of a rapidly evolving market. For example, Borland lacks support for SaaS delivery, and most tools are aimed at Windows or Linux-based developers.

Strengths
  • Borland's core functions for requirements, change management and quality management are very rich, and the products have all been proven to scale.
  • Borland provides good customer support and connection to technical teams to aid in the implementation of products and customer-driven product evolution.
  • Borland supports agile, iterative and waterfall development, including the ability to run mixed-methodology projects.
Cautions
  • Product documentation is inconsistent, leading to challenges with product configuration and adoption.
  • The three core ADLM products (StarTeam, Silk Central and Caliber) do not currently have a consistent integration API, look and feel, or platform support.
  • Sales and marketing execution have been uneven, but are improving.

CollabNet

We have rated CollabNet as a Leader in the ADLM market, with a broad market presence and functionality, and a strong play in open source. Although the product line is relatively small, with four core products, some product elements overlap; users note that the tools deliver ease of use while also scaling to support very large organizations. CollabNet has become even stronger in software change and configuration management (SCCM) since it expanded into distributed version control systems (DVCS). This strength is helping the vendor replace other legacy tools in user landscapes.

CollabNet's roots are in cloud delivery and it now generates 40% of its revenue from cloud. It is also expanding into DevOps tool chains, with solutions for automated builds and deployments. The products support six languages and are Section 508-compliant. While users are happy with the cloud platform, they have noted some issues with the on-premises tools. However, we expect that recent support for functions, such as Lightweight Directory Access Protocol (LDAP), will ease some of the administration challenges.

With its baseline of subversion support and development, CollabNet has a large customer base, and its largest site has more than 70,000 users. However, as with some of the other up-and-coming solutions in the agile space, the vendor still faces challenges as it learns to scale.

Strengths
  • CollabNet has a large customer base and proven scalability, and has shown a strong ability to execute on acquisitions to adapt to market changes.
  • It has the best usage of open source among the vendors included in the Magic Quadrant, including its Subversion (SVN), Git, Gerrit and Jenkins, and integration to Black Duck Software for identifying reused code and aiding in license compliance.
  • TeamForge provides a single integrated environment supporting planning, execution, change management and collaboration.
  • Tracker functionality and integration support is strong, enabling traceability and mixed-tool use.
Cautions
  • Acquisitions have created overlaps, which are being reduced with incremental migration to TeamForge.
  • CollabNet lacks a direct presence in South America. It lacks some of the traditional enterprise licensing and support models that large-scale users are accustomed to, and user management has been a challenge for users.
  • While CollabNet delivers in an agile fashion, some releases lack expected levels of quality and finish.

Hansoft

We rate Hansoft as a Visionary for its recognition of the challenges involved in managing product delivery in large agile or mixed-mode projects. The vendor came to the market in an interesting way; a set of developers who worked in game development founded it. These projects often have different workflows, resulting in a mix of agile/lean and more-traditional project styles (code, art and platforms). This drove Hansoft to deliver the best mixed-model development planning and execution solution available in the market.

Although the product is still relatively new and Hansoft is one of the smaller vendors Gartner covers in terms of revenue, the product has been proven to scale with multithousand concurrent user installations. However, we believe that the adoption of Hansoft will tend to stay among small to midsize teams and companies, and will lean more toward customers located in North America (its largest current customer base) and Europe than toward those in other global regions.

Strengths
  • Hansoft has strong planning and project management facilities, including project portfolio support. It also has the ability to work with varied project management styles.
  • Customer support and interaction, like in many smaller companies, are strong.
  • The vendor's background and focus have equipped it with a strong understanding of the challenges faced by software product organizations.
  • Users cite product performance, flexibility and ease of use as strengths.
Cautions
  • Integration with other life cycle tools is still weak; however, Hansoft is working with third parties to address this issue.
  • Outside the core planning of projects and work items, Hansoft's product feature set doesn't cover the breadth of ADLM; it lacks requirements management and quality management components.
  • Reporting and extensibility features in the product are limited. As the vendor grows, it will need to fund a balance between new features and bug fixes.

HP

We rate HP as a Challenger, with a strong customer base to work from and many interesting new capabilities. However, we are concerned about its ability to keep pace in a rapidly changing test and quality market, while also trying to broaden into so many fields.

HP continues to push forward by broadening its portfolio. This includes the introduction of HP Agile Manager, plug-ins to Eclipse for the HP Anywhere cloud platform and active participation in the Eclipse Orion project. HP's primary success for expansion comes from existing customers extending via HP ALM for requirements management.

HP has a global presence and proven scalability, with installations of as many as 40,000 users, support for 10 languages and a rich partner ecosystem. Because the vendor spent so many years focused wholly on the quality management market, it has strong integration APIs, supporting integrations with many other products. HP's professional services division continues to expand its transformation services, including agile at scale (see "Agile Teams Need Application Architects").

HP's products extend beyond the SDLC to include demand and portfolio management and architectural governance, along with business value metrics, to align teams with business goals. HP is also bridging to delivery and service management.

Strengths
  • HP has a dominant position in the software quality and life cycle support market, from portfolio through to delivery and operations.
  • HP's Application Lifecyle Intelligence (HP ALI) provides leading traceability and integration functionality.
  • The vendor has proven scalability and a strong partner network, providing enhanced functionality and implementation support.
Cautions
  • Unlike IBM or Microsoft, HP lacks a dedicated developer network.
  • Inconsistent user interfaces between products are gradually shifting toward the look and feel of HP Anywhere.
  • HP's agile tooling and experience in the market are less mature than market leaders, and it needs to find a unique connection to the market.

IBM

As one of a few vendors with credible offerings in almost all the subcategories of ADLM, IBM is a Leader in this Magic Quadrant. IBM Rational, which has a strong global presence, is the largest vendor in the market in terms of its installed base and ADLM revenue. IBM Rational structures its solutions for three distinct customer domains: IT, including independent software vendors (ISVs) and system integrators (SIs); systems (complex and embedded); and enterprise modernization (host systems).

Although the picture may sound clear, we still find that customers are often confused about product directions and the resolution of product overlaps. IBM is taking steps to address this via prescriptive practices and shifts in packaging and licensing. Although the products have been working toward more standardized user interfaces and technology back ends, consistency is lacking, creating integration challenges. In addition, quality can be inconsistent across the product line.

IBM is also taking a lead in pushing beyond the traditional ADLM boundaries to fill out the toolset in support of DevOps practices. Most recently, we have seen this in the acquisition of UrbanCode. We also believe that a key value of IBM is the positioning of IBM Global Services to work strategically with organizations from the executive level down to aid in transformation to mobile, cloud and agile, as well as other major shifts.

Strengths
  • IBM's broad portfolio provides coverage of the widest variety of functionality and platforms. It also supports a broad set of use cases, including large-scale, distributed teams, product and embedded system development.
  • IBM has the ability to scale to meet the needs of large and complex technology and business transformation initiatives in any global region.
  • The vendor can support a broad set of practices and development life cycles, including agile, and has made a strong push in integration via OASIS Open Services for Lifecycle Collaboration (OSLC), improving integration to other ADLM tools.
  • IBM accommodates legacy product users in product road maps, providing support and transition paths.
Cautions
  • IBM has a complex, overlapping product line that historically complicates contracts, adoption and decision making. In general, this increases per-seat costs — token-based pricing provides flexibility.
  • IBM products can involve a steep learning curve, and installation and startup costs; hosted offerings relieve installation and management costs.
  • Because IBM must accommodate a diverse installed base and legacy products, its speed of overall product innovation may lag behind less diverse companies in the short term.

Inflectra

We rate Inflectra as a Niche Player, with a solid integrated, affordable set of solutions for managing project and program information, from department to enterprise levels. Although the core products — SpiraTest and SpiraPlan — are an integrated suite offered in bundled form as SpiraTeam, they are designed with an open architecture supporting integration to competitive products. The tools are available either as licensed installations or SaaS.

Similar to other smaller companies in the ADLM space, the overall breadth of solution lacks the functional breadth of Leaders. However, the vendor is somewhat unique in having an integrated customer support package (KronoDesk), enabling project time-keeping support for product-oriented organizations, as well as for those delivering contract services. Inflectra has focused on using components that work well on touch interfaces.

The vendor continues to grow consistently. It has an established global presence, with a direct presence and partner presence in all major geographies. The products support 10 languages, but are not Section 508-compliant and are General Services Administration (GSA)-listed. Inflectra is an appealing alternative for midsize organizations, especially those focused on the delivery of consumer-facing software.

Strengths
  • Inflectra delivers feature-rich planning and management in a single-product architecture.
  • Product adoption is benefited by simple deployment, together with reasonable pricing. Customers also note that the product is easy to learn.
  • Customers report that Inflectra's support and responsiveness are very good.
Cautions
  • Inflectra has not proven its products on an extremely large scale. The vendor's largest implementation has been for 1,000 concurrent users.
  • The vendor has completed a limited number of product integrations.
  • Although the product is well-integrated, it is challenging for customers to share artifacts across projects.

Jama

We have rated Jama as a Niche Player. Its entry into ADLM comes primarily through the process of defining and managing requirements for product-focused teams. Jama's product drives traceability from requirements to validation, with support for planning and management of tests. The focus of the vendor has been on ease of use and collaboration, and a key capability is Jama's Review Center functionality.

However, the product lacks support for project planning and tracking. As a result, customers should use it to support ideation and requirements workflow in product-centric delivery organizations that already have solutions for project planning and execution. Jama integrates with other life cycle partners to fill the gaps.

The vendor has some large user sites, but customers have noted that query performance can suffer at times. Company growth has been solid, but overall market awareness is still developing. In addition to text-based requirements, Jama offers support for various models (workflow, use case) and wireframing, but it does not provide simulation or generation of test cases from use-case models. The vendor must take care to stay focused as it continues to grow.

Strengths
  • Jama delivers ease of use that is apparent in viral adoption in requirements management, an area that has often failed to overcome the incumbency of Microsoft Office.
  • Often, tools around requirements management readily become shelfware, but Jama seems to have broken that mold. Migrating traditional Microsoft Office users can still be a challenge.
  • Jama provides support for collaboration across teams. This drives resolution and management both of what is being built and the overall change process.
Cautions
  • Jama's product does not provide full coverage of ADLM functions, but does provide integrations to perform these functions, including Microsoft Team Foundation Server (TFS), Atlassian Jira, Rally and IBM Rational Doors.
  • The vendor's lack of visibility into product strategy and direction, as well as limited transparency in pricing, leaves customers wondering at times.
  • The collaboration model can yield a high volume of email messages (this option can be turned off).

Microsoft

Microsoft is a Leader in the ADLM market with a strong customer base and partner base, together with a solid stream of innovation. Microsoft offers one of the broadest sets of ADLM functionality available in the market; second only to IBM. Since delivering the Azure-based version of Microsoft TFS, the vendor has moved to a consistent release train, moving new features first to the cloud-based versions and then into on-premises releases.

In the past 12 months, Microsoft has become more open to supporting non-Microsoft technologies by embracing broadly adopted open-source technologies. The first push was to enable the use of Git as a source repository and to provide it with tooling equal to TFS's native capabilities. This support also includes Microsoft actively committing to the Git community. Microsoft also added support to its testing platform by supporting Selenium WebDriver so that one set of test scripts will now support execution on Internet Explorer (IE) or any other browser. These additions are removing challenges experienced by organizations targeting diverse technology stacks.

Going forward, the challenge for Microsoft is more at the platform adoption level: Azure, Windows Mobile, Windows 8 and its successors. Historically, Microsoft has used tools to drive a vibrant developer community that, in turn, delivered the broadest selection of software solutions. Microsoft is playing catch-up in the mobile market, placing it in an unfamiliar location where it delivers solid tools, but has to motivate developers.

Strengths
  • Microsoft has a clear strategic direction and offers ADLM functionality that is easy to implement.
  • Its developer network provides a significant pool of training materials and access to software.
  • Microsoft frequently rolls out additional and enhanced functionality in an agile fashion via SaaS, but uses a less frequent cadence with on-premises installations to avoid disruptions.
  • Microsoft has demonstrated a strong ability to make use of agile SaaS delivery and traditional release cycles.
Cautions
  • The shift toward mobile as a dominant platform creates a gap of relevance for Microsoft.
  • The vendor lacks a stand-alone requirements management approach; instead, it takes an enhance-and-integrate Office approach.
  • Microsoft lacks the agile depth of pure-play vendors around project portfolio analysis, and management or support of Scaled Agile Framework (SAFe).

Parasoft

We have rated Parasoft as a Visionary, with a focus on driving productivity through effective software quality practices. Its ADLM product strategy has two distinct facets. The primary angle is a software quality platform designed to enable IT organizations to execute automatically on practices designed to prevent defects and enable robust continuous testing in conjunction with ALM platforms. The second angle is aimed at safety-critical or regulated markets as a "self-contained" ADLM solution that enables tight quality traceability and end-user accountability. Parasoft directly supports compliance with a number of standards, including the U.S. Food and Drug Administration (FDA), Motor Industry Software Reliability Association (MISRA) and Payment Card Industry (PCI), and also provides metrics for measuring compliance with SLAs in outsourced development. Much of this support comes in the form of tools to aid in testing and source code analysis. However, Parasoft Concerto, a planning and execution product, also carries this theme, and supports agile and mixed process.

The product is available in three languages and has been proven to scale to more than 3,000 seats. Parasoft spent the past year building stronger partnerships and reworking its North American sales organization. We believe that its increased focus will be positive for users and the vendor.

Strengths
  • Parasoft provides support for compliance for several industry standards.
  • Its product set brings metric- and policy-driven decision making to continuous integration practices to drive quality improvement and reduce risks in continuous delivery.
  • Parasoft also complements many ADLM and quality management solutions with service virtualization and lab management functionality.
Cautions
  • Parasoft plans to contain rather than expand roles, and users will need to make use of integration for non-quality-focused functions.
  • The vendor brings new functionality to market fast and with good quality, but new features sometimes lack the polish that may come from slower development and introduction.
  • Parasoft has a relatively new sales team (supported by an experienced technical team) that generally executes well, but at times leads to uneven customer experiences.

Polarion Software

We have rated Polarion Software as a Visionary because of its focus on product delivery and system development, with a strong single-platform toolset that covers the entire life cycle and promotes traceability and compliance with International Organization for Standardization (ISO), FDA and many other standards. Additionally, Polarion is certified by Technischer Uberwachungsverein (TuV). The platform supports highly collaborative team development with agile and traditional approaches. However, support for agile is still not state of the art, although it is progressing consistently.

Although Polarion ALM has good support for integration and is extensible in a number of ways, the API is not well-documented and users note that extensions can break during upgrades. The vendor has partnered with agosense to improve this. Polarion Software's greatest market strength continues to be in European markets, with steady growth in other geographies (particularly the U.S.). The product can be localized, but ships in English, Chinese or French. It also integrates with Eclipse and Visual Studio and a wide variety of other tools around the life cycle. The largest installations of the product support 3,000 users.

Strengths
  • Polarion Software's single integrated product offers flexible workflow and support for large distributed teams.
  • The vendor provides support for mandated process compliance and a strong model for traceability, simplifying reporting.
  • Polarion Software has created strong integration capabilities for round-trip requirements to Microsoft Word, which enables users outside the system to participate in the review and acceptance process and overcomes a traditional barrier to requirements management tool adoption.
Cautions
  • Polarion Software's presence outside Europe is still in development, but growth in the U.S. is strong.
  • Because of the breadth of the product and its flexibility, the initial learning curve can be steep and the vendor does not have the breadth of agile transition services that agile-centric providers do.
  • Users note that updates can break integrations and extensions, and that documentation in this area is weak.

PTC

We rate PTC as a Visionary that aligns with a market need for greater integration between product life cycle management (PLM) and ADLM. This targets the products at organizations with complex process support. These include organizations involved in device and embedded development, which requires coordination between software and hardware teams and the need to deal with variant complexity. This also means that PTC delivers tools that support mixed development in agile and non-agile processes.

PTC's agile support isn't as natural as some of the pure agile tools, but has a strong backbone of support for distributed teams, as well as for the coordination of activities around the life cycle. Overall, the vendor's philosophy is one of visibility and control over software development processes and projects. PTC's Integrity product line supports integration with several products, including various elements of Microsoft Office that enable offline editing of artifacts. The largest installation supports more than 15,000 users. PTC offers direct sales and support around the globe, and is developing its partner ecosystem. The product is available in both English and Japanese.

Strengths
  • PTC's key strength lies in its support for compliance, distributed teams and variant complexity.
  • There are integrations to engineering-focused tools, such as MATLAB and Simulink, to support product delivery organizations.
  • PTC Integrity delivers strong requirements and change management tools, and integration to Microsoft Office.
  • The product is delivered as a single, unified platform, but functional components can be purchased separately.
Cautions
  • PTC does not support a SaaS delivery model.
  • The vendor does not offer support for DVCS or many other source management repositories.
  • PTC's agile tools are still emerging and show a legacy of traditional process-driven development.

Rally

Rally is rated as a Leader in this Magic Quadrant. It is deeply involved in agile software development and has executed on the acquisitions of Agile Advantage (to strengthen its portfolio planning capabilities) and Flowdock (see "Rally's Flowdock Buy Continues Trend Toward Collaboration in Development"). The vendor's AgileZen business unit has also added full support for iPad.

Rally practices lean development to introduce new functionality and evolve its products. In addition, the extensibility of the tool has resulted in a broad set of user contributions. Rally's philosophy and delivery target highly collaborative development, and we expect that the vendor will continue to strengthen support for driving agile practices into the business. Rally is a strong backer of SAFe and continues to have a strong service division and partnerships.

The product is only available in English. However, it is sold and supported globally, both directly and through a partner network, and Rally has been building a strong European presence since going public. The vendor lacks Section 508 compliance. Although the products have strong support for agile processes, support for more traditional processes is marginal, relying on kanban boards to represent things such as waterfall phases.

Strengths
  • Rally can pull together a unified view across teams and via integration to provide a transparent view of every aspect of a development project.
  • The vendor's management team continues to project a strong vision and succeeds at execution with consistent product delivery.
  • Rally's integration services and customization provide a solid platform to pull together other life cycle tools and enable users to shape the product to meet their needs.
Cautions
  • Rally's core products are available only as SaaS or appliances.
  • Support for traditional project methods is marginal. For this reason, Rally is best-suited to organizations committed to agile.
  • The vendor has grown quickly and will be under increased scrutiny regarding financial performance as a public company. As it continues to grow globally and in product mix, we believe this will create challenges to incorporate many parts into one whole and to meet a diverse customer base.

Serena Software

We have rated Serena Software as a Niche Player. It has a broad portfolio covering both mainframe and distributed systems, and spanning from ADLM to release and operations. With a good global presence, the vendor and its products are mature, with localization for 11 different languages. Serena Software's products fit best for those organizations with strong requirements for process management, auditing and reporting.

A core element of its strategy is the ability to support a broad set of tools and processes providing integration and orchestration utilizing Serena Business Manager (SBM). The vendor uses these capabilities, together with other core products, to build connected process solutions. A traditional strength for Serena Software has been in SCCM, with a strong customer base for both distributed and mainframe development. However, like other distributed platform SCCM vendors, its PVCS product line has been under pressure from open source (e.g., Git and SVN). While we believe Serena Software has fallen behind in the agile planning and requirements management segments, the vendor plans significant updates during the next year, leveraging its SBM platform.

Strengths
  • Serena Software's key strength lies in its product openness and integration to other vendor solutions, as well as in its support for integration capabilities between ADLM and release.
  • The vendor has a strong product support organization and a good history of meeting user needs.
  • This vendor offers a broad coverage of the SDLC, process types and platforms, and is well-suited for those using a mixed development process.
  • Serena Software's orchestration message provides an opportunity to bridge between the development team and management, and provides sound support for regulated processes.
Cautions
  • SilverLake, the private equity firm owning a majority interest of Serena Software, is currently seeking to sell it. Should a change in ownership occur, based on similar experiences with other vendors that Gartner follows, we speculate that this could cause changes in the team and product investments.
  • Serena Software has a broad product line, and some users note that release quality can be inconsistent.
  • Users find product implementation to be complex and that product interfaces are inconsistent.

Telerik

Telerik is a relatively new player in the ADLM market and is rated as a Niche Player, delivering two components: Test Studio for quality management and TeamPulse for project management. The vendor's focus is on agile processes with rich functionality for planning, backlog grooming, reporting and collaboration. Telerik provides full bidirectional support to Team Foundation Server.

While the company has a good global presence, it is still relatively small; in terms of ADLM revenue, it is the smallest in this research. It also has smaller team sizes using the product than its competitors. Its products are Section 508-compliant, but not GSA-listed. They are also only available in English.

The product has a simple pricing model and a unique "customer portal" that enables organizations to gather feedback from external users. The product is well-suited to ISVs, and product-centered, developer-focused teams.

Strengths
  • Telerik's customer support is fast and hands on; in addition to the provision of a significant amount of online materials, engineers get online with users and walk them through tool use. This enables users to know their input counts on future product directions.
  • Telerik customers benefit from solid quality control with users reporting that releases are reliable.
  • Solid performance and reporting are an advantage of Telerik products. Tool performance is fundamental to drive developer adoption and reports enable decision making.
Cautions
  • Telerik's products have limited capabilities to split or merge tasks.
  • Telerik's requirements functionality is purely user-story-based, supporting attachments and user comment threads.
  • The focus on TFS may challenge Telerik's ability to stay ahead of Microsoft's pace of change.

VersionOne

VersionOne is one of the original commercial agile tool vendors and is rated as a Visionary. It has led the market with a number of innovations, and now has several large customer installations (about 6,000 seats at the largest one) and a global presence.

The vendor delivers tools that provide tightly integrated processes designed to guide and manage the entire software development value chain from idea to delivery. The goal is to enable a change-friendly development approach that enables teams to adapt to rapidly changing business priorities and plans.

VersionOne values simplicity over complexity, and customers note ease of adoption. The vendor provides a broad set of materials for adaptation of its tools and agile processes, including e-learning, miniconferences and the Agile Sherpa community portal.

VersionOne has continued to advance the product. During the past 12 months, it has added TeamRooms, Conversations and Ideas, as well as enhanced diagramming to improve the ability to visualize relationships and dependencies. Currently, the product is only available in English and it is not Section 508-compliant.

Strengths
  • VersionOne offers flexible licensing and is available via SaaS or on-premises delivery models.
  • It has a broad set of prebuilt integrations, including portfolio views and support for SAFe, and has achieved a good balance between being easy to adopt, yet able to scale.
  • Agile transition support includes personalized coaching to drive tool use and covering everything from the basics to advanced practices.
Cautions
  • VersionOne currently only has offices in the U.S., and relies on partners and online interactions to support other regions.
  • Support for nonagile processes is limited, as is functionality around quality management and SCCM.
  • Administration can become cumbersome in larger organizations.

Vendors Added and Dropped

We review and adjust our inclusion criteria for Magic Quadrants and MarketScopes as markets change. As a result of these adjustments, the mix of vendors in any Magic Quadrant or MarketScope may change over time. A vendor appearing in a Magic Quadrant or MarketScope one year and not the next does not necessarily indicate that we have changed our opinion of that vendor. This may be a reflection of a change in the market and, therefore, changed evaluation criteria, or a change of focus by a vendor.

In this edition of the Magic Quadrant, we added vendors participating in the product delivery segment as we see this as a growing category. We removed three vendors to make space. Although these three vendors still provide solid support in their respective niches, they have not been as prevalent in Gartner client inquiries.

Added

  • Hansoft
  • Jama
  • Telerik

Dropped

  • TechExcel
  • ThoughtWorks Studios
  • Rocket Software

Inclusion and Exclusion Criteria

To be included in this Magic Quadrant as an ADLM tool, offerings must:

  • Generate at least $10 million in ADLM revenue
  • Report having at least 600 customer installations
  • Support the following management domains:
    • Requirements management — the ability to define requirements, manage changes and trace dependencies
    • Quality management — the ability to define test cases and manage defects
    • Project planning and management — the ability to define and assign work items and track and report on status
  • Facilitate distributed team activities
  • Manage change process workflows from initial change requests or requirements through build and turnover for release
  • Support federated sharing or central storage of metadata for the development resources and processes
  • Support custom reporting and custom integrations beyond those of the vendor
  • Be generally available
  • Have at least three reference clients in production
  • Be distributed in multiple national markets
  • Make commercial support available

We have excluded ADLM offerings that are package-focused. Specialized ADLM facilities, emerging for major ERP environments, such as those of SAP and Oracle, need to be considered separately. SAP clients should explore SAP ALM and offerings from companies such as Realtech, Revelation Software Concepts, Panaya and IntelliCorp, as well as the use of SAP Solution Manager. Oracle users can explore companies such as Dell-Quest Software, Unitask Software and Phire. No single offering is capable of addressing ADLM needs in both custom-developed and packaged environments at this time.

We also excluded offerings that required a significant amount of custom programming services to complete. A number of products have some ADLM capabilities, but failed to meet one or more of the above criteria, so they were excluded from this Magic Quadrant.

Evaluation Criteria

Ability to Execute

Product execution is critical in this market, and Leaders are able to deliver new functionality at a consistent pace with consistent user interfaces. In user surveys and a growing number of calls, we have found a desire for lower-cost alternatives, open source and/or SaaS or token models to support flexible teams. Product execution also comes from products with good integration strategies to other ADLM tools (both from the vendors and competitors). In addition, product execution suffers, with tools that are hard to implement and challenges with upgrading to new versions. Therefore, leading execution means delivering intuitive tools with a consistent delivery cadence with strong integration capabilities, and using innovative pricing models.

These changing pricing models are effecting a change in what sales execution often means. Leaders have an ability to execute strong proofs of concept remotely, deliver tools that are intuitive and support a variety of pricing models, including SaaS, token and enterprise. We note that some organizations are outstanding at initial deal execution, but may be challenged in the larger enterprise context, while some of the traditional vendors are more challenging to work with initially, but have the breadth of global presence and an ability to execute at the enterprise level.

The power of social computing has steadily become the driver for how marketing execution is measured. This is also related to overall customer experience, with both working in concert to deliver a clear message and execute with a solid product. The most effective marketing is having products in the market that users love and talk about. No degree of marketing can overcome poorly executed products. We look at overall social presence and user sentiment. This also relates to marketing execution, in which the ability to create the organization's image and brand has become less important than delivering a product, clear direction and user value. Most users have a wide variety of products on-premises (including tools for both custom and package development), which means that partners and support for integration standards are key execution points.

We weighted overall viability, responsiveness and operations as Medium, as they support the High criteria and create indicators for execution effectiveness. Viability is a challenge to look at, depending on where a company is in its overall life (private or public), but we measure overall market share positions, global presence and strength of partnerships.

Responsive companies are able to pivot effectively as the market shifts. Too often, this means trend washing — simply applying the latest buzzwords to the marketing message. Companies that are responsive are able to deliver new products and services in step with market needs, and to provide outstanding customer service.

Operations are key because they tie in to responsive companies and the ability to actually deliver software. For SaaS companies, it is the ability to scale, to be stable and to deliver cost/price advantages. It is impossible to be an agile hypocrite, preaching one direction while actually operating in a different one. The has been a difficult challenge for many of the traditional vendors, which had well-oiled traditional delivery pipelines, but no real experience with agile, continuous delivery.

Other Considerations

Many of the vendors in this Magic Quadrant are still challenged in terms of their ability to support large-scale, global enterprise use. We mainly note this with newer companies, primarily in the agile or product delivery space. All the vendors included in this research provide good support in the U.S.; however, organizations that need a global-enterprise-capable vendor should focus on IBM, Microsoft, HP, Serena Software, Borland and Parasoft, with Atlassian, Polarion Software and PTC being considered depending on market segment or specific country mix. A key element of Ability to Execute for vendors as they mature is the ability to grow their global businesses through partnerships, local account management and language support.

Section 508 Compliance

Tools in the ADLM space are mixed in their support for Section 508. This is especially true for browser-based tools, which find challenges in the facilities exposed by various browsers and rich-client browser toolkits. In addition, as tools enable visual and drag-and-drop-oriented facilities, such as task boards, the ability to use screen readers and provide compelling interaction models becomes problematic. For entities that require Section 508 compliance, or that are seeking tools that support accessibility, the best support is currently offered by Kovair, Microsoft, Parasoft and Polarion Software. Rocket Aldon, Digite, Inflectra and Serena Software provide partial support.

Table 1. Ability to Execute Evaluation Criteria

Evaluation Criteria

Weighting

Product or Service

High

Overall Viability

Medium

Sales Execution/Pricing

Medium

Market Responsiveness/Record

Medium

Marketing Execution

High

Customer Experience

High

Operations

Medium

Source: Gartner (November 2013)

Completeness of Vision

The key areas of Completeness of Vision are the ability to understand the market from two perspectives:

  • Many vendors still provide a "complete" ADLM stack to manage or govern the activities of defining, developing, accepting and delivering solutions, which provides value in traceability and reporting.
  • Most vendors, however, won't be supplying the only ADLM tool in use, so they must have an integration strategy.

We rated the vision of vendors by looking at how complete their offerings were for the various use cases they participate in, as well as looking at the breadth of markets served. Because of the strong relevance of agile, we placed the most emphasis on how vendors are driving agile vision in the areas of collaboration, scale (both in size and roles supported), and connection to portfolio management and application release. These elements feed the scoring for a vendor's innovation abilities.

Vendors without an understanding of market trends in practices and the evolution of delivery platforms will slowly find themselves pushed into a corner. Once a vendor has a solid understanding, the ability to innovate (which creates differentiation) in technology, as well as the offer strategy, are critical. Gartner's client and user surveys demonstrate a desire for greater visibility of product direction, more frequent and consistent delivery of new features (maintaining compatibility), and a need for leadership and practices, not just tools that will help enterprises transform themselves.

Offering (product) strategies are evaluated by looking at the flexibility of delivery: on-site, hosted or managed service provider. We also look at the packaging with transformation services, delivery of materials and services to aid in making transitions and successful use of the products. Are the products unique or well-suited to specific markets, and is the vendor able to deliver the right focus, partnerships and packaging?

Looking at the medium influences on vision (marketing, sales, vertical/industry strategy and business model), we assess these by looking at channel strategies that drive adoption, the ability to deliver both product and content, and/or a focus on specific verticals (generally, this comes out as the use cases supported). It should be noted that vertical or industry focus may be high for your organization and it is difficult for this to be made clear in a vendor's position. Most vendors are focused on IT and customer-facing software. If your needs are in regulated industries, embedded or system development, and/or product delivery, we have tried to also note various abilities in our vendor descriptions in this Magic Quadrant.

With many more vendors participating in ADLM than Gartner is mapping in a subsegment of the market, vertical or industry strategy is less important from a Completeness of Vision perspective. However, it will become more of a differentiating strategy for vendors that end up servicing well-defined market segments, rather than trying to battle in the overcrowded general market. Good examples of this are tools focused on product-centric development teams. These teams tend to have needs for mixed project styles, as well as variant complexity or regulatory needs that must be met.

Business model innovation is projected through the ability to utilize new models, such as token-based and SaaS models to provide flexibility in licensing. Innovators are also looking at ways to position their offerings as platform as a service (PaaS; see "Hype Cycle for Platform as a Service (PaaS), 2013"). This enables these vendors to deliver additional value through partners, thus extending the core products. We look at the partner network and where extensions have been provided. Sales strategy is also measured by looking at supported channels and how the vendor supports global service providers.

We weigh geographic strategy as Low due to the vendors' ability to service customers remotely via electronic means. We look at the number of languages supported, the points of presence (for SaaS tools), and where partners are available for sales and support.

Table 2. Completeness of Vision Evaluation Criteria

Evaluation Criteria

Weighting

Market Understanding

High

Marketing Strategy

Medium

Sales Strategy

Medium

Offering (Product) Strategy

High

Business Model

Medium

Vertical/Industry Strategy

Medium

Innovation

High

Geographic Strategy

Low

Source: Gartner (November 2013)

Quadrant Descriptions

Leaders

Leaders have established strong market positions, as shown by breadth of adoption, global deployments and integration of other products. They generally have breadth across the development life cycle, and deliver market-leading functionality in one or more functional areas. Leaders have strong capabilities in agile practices, including driving portfolio management support and support for enterprise agile capabilities, such as SAFe and Disciplined Agile Delivery (DAD). This demonstrates the ability to deliver software in a more agile fashion, as well as the provision of tools and services designed to aid agile transformation. If vendors want to remain Leaders, then it is increasingly important for them to support more-agile delivery schedules and diverse channels.

Challengers

Challengers have executed well in a specific use case or market, but tend to have limited breadth or market penetration across the life cycle. These vendors tend to have well-established partnerships and a solid global presence. In general, Challengers may have been innovators at one point, but are not seen as driving the application development market. They have a strong presence in a portion of the market, but have not yet created a strong presence in the entire market.

Visionaries

These vendors have a strong vision in a specific category that potentially disrupts the market, but have limitations in the overall breadth of solution or geographic coverage (in terms of language, support and partners). Because of their focus, organizations will often find that supply chain solutions from Visionaries are easier to adopt. They may adopt these solutions virally. Customers also may, however, swiftly replace these solutions with the next hot product.

Niche Players

Niche Players either have a strong market niche (for example, product development) or a significant legacy footprint. They may also provide strength in a functional area of ADLM, but may not be as strong in other areas or may have turned their focus to other adjacent areas (such as DevOps), thus slowing product enhancements to the core product lines. In some cases, these vendors may be more mature and able to support customers than some of the Visionary providers.

Context

Organizations are showing more interest in integrated ADLM because they want an integrated solution, and not simply because they want to acquire a single vendor's solution stack. However, customers rarely start with a vision of massive enterprise deployment; instead, they seek to solve specific problems. As a result, most vendors have several overlapping ADLM solutions, rather than one. Although this may not seem like a best practice, it is a practical practice that reinforces the need for tools to work together, and for solutions that bridge the silos with workflow and reporting.

Most organizations will continue to need a mixture of application development solutions. The more diverse the business and technology, the more diverse the tools an organization will need to support delivery. Evaluations should place a high value on tool-to-tool integration mechanisms, using XML and REST or broadly adopted proprietary mechanisms.

Different markets or users have varying needs, which can include strong needs for:

  • Traceability and reporting for compliance
  • Support for collaboration across roles brought on by the adoption of agile practices or distributed teams
  • The ability to deal with complex requirements and variant complexity

More discrete needs are converging and sometimes clash. This clash isn't just at a tool level; we also see divergent practices within organizations. This is further complicated by a broader number of roles that are becoming integrated with the rise of DevOps practices and tool chains.

Many of the providers in this Magic Quadrant provide build management and continuous integration facilities. Some have also built or acquired support for application release automation (ARA; see "Know the Application Release Automation Vendor Landscape to Shortlist the Best Vendors for Your Organization"). This will create some confusion in the market as vendors seek to be on top of the next big buzzword. Many are already seeking to incorporate DevOps — the confluence between development, build and release and IT operations release planning and management — into the next generation of ALM.

Although the drive toward continuous integration and delivery practices will affect the solutions in the market, organizations will need to separate the practices from specific tool categories. When considering agile tools, they will need to evaluate the provider's agile development and delivery, geographic distribution of teams, easy-to-use and compliant process-centric ADLM, product delivery, and integrations.

Our Magic Quadrant research has shown that the breadth of use of ADLM tools is expanding in organizations. Last year, we saw that 58% of use was for multiple roles on a project or team level. This year, that level has dropped to 43%, while enterprisewide adoption has grown from 37% to 52%.

Once the organization has selected a tool, it will also need to establish a team that will manage integration architecture, implementation and training. This will help to ensure that the ADLM tool is effective. This same team should play into the definition and rollout of DevOps practices and should include cross-discipline input.

Adoption of ADLM tools sometimes leads to challenges in properly scaling the tools and dealing with user administration. It can sometimes place the capabilities of the smaller providers under pressure to provide the necessary support. Other common challenges with tools include:

  • Integration (while tools provide the basic facilities, the pictures are often better than reality)
  • Upgrades that break existing work or are a challenge to execute
  • High initial learning curves, and a variety of challenges with pricing and licensing

Organizations are under pressure to accelerate the speed of delivery of increasingly complex applications, while improving overall productivity and quality. They are adding packages, business process management suites, externally provided services and other new delivery vehicles to their range of options. These applications then combine with legacy technologies, often built by virtual teams spread around the globe.

Audit and oversight demands continue to grow in both regulated and unregulated environments. The need for greater governance often adversely affects productivity. Efficient coordination and automation of the delivery process require new, collaborative approaches to the planning, measurement, execution, control and reporting of activities. These new approaches are what differentiate ADLM tools, and what make ADLM processes vital to leading-edge development activities.

ADLM suites have promised improved automation and integrated approaches to the delivery of applications, but they have often fallen short of the vision. Users are seeking ways to coordinate work and share data across phases and activities, which include requirements definition and management, different testing activities (including test case management), software changes, and configuration management.

However, vendors often have process enactment tools for the various major activities and are seeding solutions that don't require a rip-and-replace approach. They are resisting the tendency of product suppliers to apply the ADLM term broadly to include functions focused on project execution. They recognize that the suites of applications available during the past 10 years have fallen short in their ability to maintain consistent and complete views across many process steps.

Depending on their philosophy of ADLM, vendors are taking different approaches; however, many have shifted toward a federated repository approach. This allows specific implementation tools (compilers, debuggers and modeling tools) to share information about artifacts. This workflow system describes the (sometimes quite messy) sequence of activities required to design, develop and deploy the artifact. A data warehouse enables the capture of information about practices, so that they can be repeated. Although ADLM includes the management of specific phases — such as requirements, design and test — the extension of unified workflow and management across these phases is the key element. The emergence of service-oriented architectures (SOAs) is enabling this evolution through the use of XML, REST and RSS.

Market Overview

The ever-changing nature of development (including changing languages, platforms and supporting tools) drives a market that supports choices in tools. Part of this will be the evolution of what ADLM is. "ADLM" is a term focused on the SDLC, rather than the application life cycle, from inception through retirement. In particular, the emergence of DevOps as a primary driver will further push the connection between traditional ADLM and IT service management. This will continue to be a driver of acquisitions in the market.

The broad range of evolving ADLM offerings leads us to recommend a scenario-based evaluation process to narrow the list of candidates (see "Selection Criteria for Success in Choosing ADLM Products," "Toolkit: RFP for Application Life Cycle Management and Related Tools" and "Toolkit: RFI Template for Application Life Cycle Management and Related Tools").

Gartner estimates that the ADLM market will continue to grow between 5% and 7% annually through 2016. Portions of the market will remain stronger than others. Much of the growth of the products in this Magic Quadrant is in the agile planning category. There is also decent growth in the requirements area, particularly in definition tools, which only a subset of the vendors in this Magic Quadrant provide.

Gartner expects that there will continue to be a variety of ADLM options. This will include agile development teams using lighter versions of ADLM offerings; solutions integrated to package deployments; continued growth of open-source options; and an increasing number of cloud-delivered, ADLM PaaS solutions.

This Magic Quadrant covers only a small set of the vendors in the ADLM market. New vendors regularly enter the market through acquisitions or by creating new point-specific tools, as new areas or methods arise. However, there are too many discrete vendors and few that can drive long-term direction.

This will force vendors into supporting specific market niches, such as small or midsize, package solutions, legacy renewal and embedded applications. We also expect that the move toward cloud-based tools with RESTful interfaces will continue to enable and enhance the user behavior of having tools from multiple vendors.

Evidence

To create this Magic Quadrant update, we carried out a variety of activities. A key element was the use of online surveys of customers of the included vendors, utilizing Qualtrics. We surveyed 150 discrete companies in this research. In addition, we made use of Gartner research to understand market revenue and growth. We also analyzed Gartner client call data and utilized information from a variety of social media sites to understand effectiveness of execution.

Evaluation Criteria Definitions

Ability to Execute

Product/Service: Core goods and services offered by the vendor for the defined market. This includes current product/service capabilities, quality, feature sets, skills and so on, whether offered natively or through OEM agreements/partnerships as defined in the market definition and detailed in the subcriteria.

Overall Viability: Viability includes an assessment of the overall organization's financial health, the financial and practical success of the business unit, and the likelihood that the individual business unit will continue investing in the product, will continue offering the product and will advance the state of the art within the organization's portfolio of products.

Sales Execution/Pricing: The vendor's capabilities in all presales activities and the structure that supports them. This includes deal management, pricing and negotiation, presales support, and the overall effectiveness of the sales channel.

Market Responsiveness/Record: Ability to respond, change direction, be flexible and achieve competitive success as opportunities develop, competitors act, customer needs evolve and market dynamics change. This criterion also considers the vendor's history of responsiveness.

Marketing Execution: The clarity, quality, creativity and efficacy of programs designed to deliver the organization's message to influence the market, promote the brand and business, increase awareness of the products, and establish a positive identification with the product/brand and organization in the minds of buyers. This "mind share" can be driven by a combination of publicity, promotional initiatives, thought leadership, word of mouth and sales activities.

Customer Experience: Relationships, products and services/programs that enable clients to be successful with the products evaluated. Specifically, this includes the ways customers receive technical support or account support. This can also include ancillary tools, customer support programs (and the quality thereof), availability of user groups, service-level agreements and so on.

Operations: The ability of the organization to meet its goals and commitments. Factors include the quality of the organizational structure, including skills, experiences, programs, systems and other vehicles that enable the organization to operate effectively and efficiently on an ongoing basis.

Completeness of Vision

Market Understanding: Ability of the vendor to understand buyers' wants and needs and to translate those into products and services. Vendors that show the highest degree of vision listen to and understand buyers' wants and needs, and can shape or enhance those with their added vision.

Marketing Strategy: A clear, differentiated set of messages consistently communicated throughout the organization and externalized through the website, advertising, customer programs and positioning statements.

Sales Strategy: The strategy for selling products that uses the appropriate network of direct and indirect sales, marketing, service, and communication affiliates that extend the scope and depth of market reach, skills, expertise, technologies, services and the customer base.

Offering (Product) Strategy: The vendor's approach to product development and delivery that emphasizes differentiation, functionality, methodology and feature sets as they map to current and future requirements.

Business Model: The soundness and logic of the vendor's underlying business proposition.

Vertical/Industry Strategy: The vendor's strategy to direct resources, skills and offerings to meet the specific needs of individual market segments, including vertical markets.

Innovation: Direct, related, complementary and synergistic layouts of resources, expertise or capital for investment, consolidation, defensive or pre-emptive purposes.

Geographic Strategy: The vendor's strategy to direct resources, skills and offerings to meet the specific needs of geographies outside the "home" or native geography, either directly or through partners, channels and subsidiaries as appropriate for that geography and market.