Intelligent ALM™ – The Prerequisite for Enterprise Agile

Agile development is a hot topic today – there’s not a tradeshow, a technology publication or development tools vendor that’s not messaging about Agile. The appeal of Agile is in its perceived ability to help organizations decrease time to market, improve the quality of delivered software and increase overall productivity.

There are plenty of examples that clearly show the benefits of Agile in grass roots and pilot implementations, but does Agile provide all those benefits on an enterprise level? If so, how do we scale Agile in an enterprise environment? Can it coexist with traditional processes? How do we build the business case for enterprise adoption through pilots and then roll out Agile to other parts of the organization?

How is Agile Different?

Agile differs from traditional plan-driven approaches in a number of ways. For example, Agile promotes a closely collaborative, cooperative team approach, where members of the team are not segregated into narrow disciplines. Agile teams are expected to be continuously adapting, refining and improving; practices as well as the product are evolved iteratively and incrementally. Gartner identifies that “Agile requires team players who are flexible and invested in the success of the team”¹.

Agile also emphasizes responding to change instead of following a plan. Lean software development phrases this as “defer commitment as late as possible, and then deliver as fast as possible”. The use of short, fixed duration iterations which end with stable working code is key to achieving this responsiveness.

Finally, Agile approaches require the customer (or their representatives, known as product owners) to be an integral part of the development team. The stakeholders and developers are all focused on the common goal of maximizing business value.

Agile in the Large

Traditional Agile approaches arose as a reaction to the heavyweight bureaucracy typically associated with large plan-driven methodologies; they emphasized a reliance on the skill and discipline of the co-located team members rather than detailed formal processes and systems. With a single small, co-located team, simple mechanisms such as sticky notes, index cards and white boards are more than adequate for maintaining and disseminating information. Face-to-face conversations suffice for answering questions and coordinating activities for these small teams.

The scale and complexity of the enterprise environment requires more support than simple tools and informal methods can provide. The Agile Manifesto states “We value individuals and interactions over processes and tools”; that is – processes and tools should be enablers to allow the team to be effective, and not an ends and means unto themselves. This value statement does not mean that all processes and practices should be eliminated, or that tools are to be avoided.

Given the emphasis that Agile places on cross-functional teams working in a highly collaborative manner, enterprise adoption of Agile practices requires such organizations to follow an Application Lifecycle Management (ALM) approach to software development. Proper ALM is the coordinated management of all development activities, emphasizing the interrelated nature of development activities and assets (which is why it complements Agile so well). Traditional phased approaches encourage the different disciplines such as requirements management, design, coding, testing and build management to be treated as distinct and relatively isolated activities. These discipline silos do not foster the cross-team communication and closed loop feedback required by Agile teams.

Implementing Agile practices within large enterprise environments involves more than just scaling the number of people working on the project. Enterprise environments introduce a number of different scalability issues: team size, distributed sites, compliance requirements, domain or product complexity, technological complexity and organizational distribution, to name but a few. Adequately supporting large distributed Agile teams working on such complex product lines requires enterprise ALM solutions. According to Gartner “Application life cycle management (ALM) tools provide critical governance, collaboration, change management and workflow functions to ensure that teams work efficiently and that corporate standards are maintained”.¹

ALM Competence

As demanded by the Agile Manifesto, ALM solutions (with their tools and processes) must support development teams without getting in the way or limiting the ability of teams to continuously adapt and improve. Suitable ALM solutions for enterprises that are adopting Agile practices must provide competence in all the key disciplines of software development, such as demand management, project management, requirements management, design, coding, testing and release management. Without this complete competence, teams suffer from a lack of transparency, stale or incomplete views of development activity, and the risk of omission or inconsistency due to needless data redundancy and manual integrations across disparate tools.

ALM Coherence

ALM for Agile enterprises must support inherently connected activities, assets and processes. Such an ALM solution needs to provide a single coherent framework and common set of interfaces that orchestrates development lifecycle activities in a consistent manner. This is accomplished through automation and enforcement of defined practices, active management of relationships between all development activities and assets, and single source of truth reporting on the development effort as a whole. Consistent change and configuration management processes should apply to all assets across the lifecycle.

ALM Adaptability

Within enterprise environments, the Agile focus on continuous improvement requires a single ALM solution that is adaptable and flexible to support continuous controlled changes to processes and practices, and different teams using different sets of practices or variations on a given practice. Most enterprise organizations will also require their ALM solution to support a mix of traditional methodologies and Agile practices, since they have existing projects and teams that may eventually transition to Agile, but cannot afford to do a ’big bang’ switchover. Enterprise environments also include a variety of point tools to which the ALM solution must integrate, such as test automation, build execution, integrated development environments (IDEs), modeling tools, and complementary systems that ALM must interact with such as help desks, PLM systems and IT Service Management.

Intelligent ALM

MKS identifies the combination of Competence, Coherence and Adaptability within an ALM solution as Intelligent ALM™. With complete ALM competence, delivered in a coherent single solution that rapidly adapts and scales, Agile can thrive in the enterprise environment. Without the ability to easily navigate all the activities and assets, the ability to refine and improve processes and practices on a regular basis, or easily switch roles (without having to change tools and contexts), the ability to follow the values of Agile in a large organization will be severely constrained.

Start Small, but Plan Big

In many enterprises, existing environments are well established and complex. The experience and skill levels of the development organization may vary widely, and a certain amount of turnover within the organization is inevitable. Implementing an Agile approach involves a significant shift in the culture of the organization and changes in behavior as well as practices. For these reasons, it is not advisable to take a ’big bang’ approach. Most enterprises will require a solution that can effectively support traditional processes as well as the new Agile processes, in order to ease transition between the two. Remember – the transition to Agile will neither happen in a vacuum, nor in a single moment of time.

Gartner recommends that organizations “Start small when adapting process change, and focus budget efforts on controlling the number of projects, rather than reducing teams and processes.”¹ To this point, start small – take on Agile within one or a few suitable projects, with the objective being to demonstrate success early. Management expects better software, faster delivery and lower costs. Use the AL M solution to automate the collection and reporting of key metrics, so that the benefits can be quantified to the rest of the organization and to aid in the Agile team retrospectives (continuous improvement process).

The initial project teams must lay a suitable foundation for demonstrating this kind of success, and then be ready to take the tested processes to other teams once the Agile initiative has been approved for expansion. Having verifiable metrics and reporting to justify the expansion is critical – well managed enterprises will not adopt Agile on blind faith. Effective ALM solutions must facilitate the transition between traditional and Agile processes (with the ability to effectively support both) and must aid in the delivery of verifiable and relevant software delivery metrics.

ALM – The Single Source of Truth

A successful enterprise transition to Agile requires an ALM solution that will act as a single source of truth for all of development, regardless of project or process. Timely feedback is a key aspect of implementing Agile, particularly within enterprise organizations. A key tenet of Agile is the shortening of feedback loops –this allows Agile to reduce risk by delivering value earlier (in the form of iterative working software builds). Transparency across the entire project and short feedback loops in the form of continuous integration build and test practices help Agile organizations to improve the quality of the software produced. The ALM system will be relied on to deliver reports and metrics that provide that feedback, and demonstrate the success of the Agile approach.

Real-time status should be automatically collected by the ALM system as a result of the development activities the teams perform; no manual extraction and collation of data in a third-party reporting system should be done (such activities do not directly contribute to “working software”, which is the focus of Agile). With an ALM solution providing a single source of truth, ongoing monitoring of progress is based on current data and reports can be generated at any time with the click of a button. Such monitoring and reporting keeps teams up to date on the status of their sprints and gives management the visibility needed to assess the health of the entire Agile initiative. Once success has been demonstrated, an ALM solution can aid in the process of transferring the tested processes to other teams in a consistent fashion.

To deliver meaningful and accurate metrics, all development activities and assets need to be seamlessly interconnected. In fact, in high-functioning Agile environments, any individual may need to work with any type of asset (user story, code, test case, etc.) as team members have the tendency to take on different roles (such as a developer taking on the role of tester). The most efficient way for the developer to work is from a single application that enables access to all of these activities and assets. This is where ALM “suites” – collections of integrated point tools – fall short. Integrations limit visibility and access to data. Only a coherent Intelligent ALM solution that houses all activities and assets (including requirements, code, components, test cases, etc.) can deliver the transparency to address the needs of Agile teams where roles are sometimes interchanged and data must be complete and available in real time.

Adaptive Automation with Intelligent ALM

The practices of Agile are meant to reduce redundancy. An ALM solution that can automate non-core activities (such as the collation of data for reporting, the computation of metrics, and the production of audit trails to address regulatory requirements) helps to reduce redundant effort and eliminates time-consuming manual processes. To truly support enterprise Agile, this automation needs to be implemented in a way that it is adaptable to process change. Agile is about adapting to change, and thus, processes will change. An ALM solution needs to be easily adaptable to changes in process, and should be able to embrace change, rather than limiting the scope of change.

Integrated ALM suites fall short in delivering automation to Agile enterprises, because integration eliminates adaptability. Making a process change across a range of integrated, disparate tools can mean days or weeks of programmatic changes. Teams spend their time trying to customize tools to extend their functionality (instead of delivering software). With an adaptable Intelligent ALM solution, processes and assets can be easily configured and re-configured as necessary.

Competence, Collaboration and Collective Memory

Agile demands a high degree of collaboration amongst team members and across the lifecycle. This collaboration becomes more complicated when teams are distributed – studies show that even the dispersion of team members across different floors of the same building can have a negative impact on Agile project success. When the team is distributed across locations, time-zones and even different companies, sustaining collaboration becomes a major challenge.

Integrated ALM suites hinder collaboration as each role is siloed in its tool usage and will have difficulty communicating across functional boundaries. For instance, a point tool that only manages changes to source code cannot deliver real transparency and collaborative capabilities to the business (because the development team still remains siloed from the Testing organization and from the Product Owners developing the requirements that map to User Stories).

Intelligent ALM solutions make it possible for even distributed teams to embrace collaboration because all stakeholders (regardless of location or role) are interacting with the same activities and assets in the same solution. All team members benefit from transparency into a project’s current status and information sharing that flows easily from state to state, member to member, sprint to sprint, and so forth. With a coherent solution, Agile teams can more consistently and effectively manage activities and assets in relationship to one another throughout the application lifecycle. A single ALM solution for all of development also streamlines transition of staff between projects or to new teams (as knowledge of the tool and processes does not change).

An Intelligent ALM solution effectively becomes a collective team memory for the entire development organization. Not only can individual teams and projects achieve continuous improvement but they can share their learning and best practices across other teams throughout the enterprise. Agile teams without such a solution will leave the business at risk when a member leaves.

In Conclusion – Scale Agile Successfully

Like any process change, cultural challenges can affect the success of enterprise Agile initiatives. Starting small and demonstrating initial success with supporting metrics will help to overcome those challenges quickly. When buy-in has been achieved, the business needs to be ready to build on and leverage that initial success.

Having an Intelligent ALM solution in place from the start enables the organization to incrementally roll out proven processes, building on a foundation of best practices. Competence across the complete range of disciplines in development ensures that all of the needed assets and activities can be managed and controlled while eliminating redundant manual effort by the teams. Coherence in conjunction with that complete competence enables collaboration that can match the demands of enterprise Agile. Adaptability enables the organization to roll-out Agile in an iterative fashion, maintaining plan-driven development processes where desired and keeping the critical traceability and audit trails in place to maintain or achieve compliance.

By planning for success from the start and laying the foundation for expansion with Intelligent ALM solutions, you can ensure the scalability of your enterprise Agile initiative.

¹Gartner Research Don’t Let Short-Term Agile Create Long-Term Pain, Thomas E. Murphy, Jim Duggan, David Norton, 7 April 2009

Source: Colin Doyle, Product Manager, MKS
"In many enterprises, existing environments are well established and complex…Implementing an Agile approach involves a significant shift in the culture of the organization and changes in behavior as well as practices. For these reasons, it is not advisable to take a 'big bang' approach. Most enterprises will require a solution that can effectively support traditional processes as well as the new Agile processes, in order to ease transition between the two."
Colin Doyle, Product Manager, MKS