Balancing Six Sigma and the Capability Maturity Model

Balancing Six Sigma and the Capability Maturity Model (CMM^®/CMMI^sm)

Challenged by the changes during the past 10 years in the importance and utilization of software by businesses, enterprises now view the cost and quality of software development differently. Whereas IS organizations formerly minimized the significance of development engineering costs as non-recurring, the current critical nature of software in a product or service offering has elevated the importance of the development process. Thus, it is essential to maximize the use of any available tools or models to increase the efficiency and effectiveness of software development.

Although two prominent approaches are available, whether to incorporate one or both solutions can be perplexing. One solution involves a Capability Maturity Model (CMM^® or CMMI^sm) as a framework for improving the methods and processes used for software product or service development and delivery. Another frequently considered choice is to implement a Six Sigma program to address quality and customer satisfaction issues. It is imperative to understand whether these approaches are compatible for joint use or should be used independently.

Six Sigma is applied in a process-specific way. Trained "Black Belts" lead process improvement projects that define, measure, analyze, improve and control selected processes. Six Sigma's results are optimized by applying the program to iterative processes (e.g., producing a consistent product or operations delivering a service to many customers). Six Sigma projects typically address process problems and produce a measurable return on investment. The Capability Maturity Models are organizational change models that reflect the steps an enterprise must take to improve its software development process capability. With five levels of maturity for implementing software development practices, each level builds on the lower levels (see Figure 1). Whereas Six Sigma projects typically are short-term (that is, three to six months), the Capability Maturity Model is applied long term across an enterprise to improve the success of application development or maintenance projects by institutionalizing practices known to achieve good results.

Six Sigma and the CMM also view quality from two different perspectives. The goal of a Six Sigma program is to reduce defects (that is, any problem with service performance or a product fault that does not meet the customer's specification) to a level that statistically relates to 3.4 defects per million instances or opportunities. For CMM, however, any defects occur in internal work products with negative ramifications to the completion of subsequent work. The fundamental premise behind the CMM is to improve the quality of a software product by improving the quality of the development process.

Maturity Level 2

Whereas CMM Level 1 enterprises demonstrate no development methodology and few processes, Level 2 enterprises exhibit repeatable tasks and processes, however, inconsistently. The CMM establishes policies, provides some documented guidance, and instantiates management practices that support and track software development project commitments. Measures at this level are limited to tracking size, effort, schedule and cost over the life of a project. Unlike Six Sigma, the CMM ensures a progression through Level 2 with policies, plans and commitments that must be followed, thus establishing a project management culture. The Six Sigma variability of project performance and lack of a defined process inhibit a statistically valid analysis in which consistent sources of defects are identified. Despite these drawbacks, Six Sigma training would be useful for analyzing development life cycles and establishing historical data for improved software project cost and schedule estimations as enterprises move to Level 3.

Maturity Level 3

Leveraging Level 2 capabilities, Level 3 enterprises improve the consistency of their organizational processes and detail methods for building and delivering applications, with training support. Best practices are institutionalized in a process description. In addition to a set of organizational process assets, process engineering groups develop an organizational measurement database. By itself, Six Sigma does not offer such elements. At Level 3, measures usually are taken at project milestones or collected and reported periodically, which is not granular enough for effective statistical analysis using the full arsenal of Six Sigma tactics and techniques. Initially, the data from the projects is inconsistent and unclear. With greater consistency and commonly shared definitions across projects, the quality and usefulness of the data improve. A Six Sigma group can inhibit the instantiation of Level 3 practices by concentrating on short-term engagements and subprocess optimization. However, using a CMM framework, trained Green Belts or Black Belts can help establish derive processes and create a measurement foundation and infrastructure that will support Level 4 quantitative management techniques.

Maturity Level 4

Level 4 is characterized primarily by measurement. Actual results of process performance on development projects are compared in a statistically significant manner with the expected results. Measurements occur at the process event level in near real time. In addition, the level of automated data collection and analysis is sophisticated. To implement Level 4, the development organization establishes and maintains baselines of process performance and capability. Based on the known capability, projects establish performance goals and use actual process performance data and statistical tools to manage toward these goals. At this level, Six Sigma techniques become extremely useful, and its training for managers and developers emphasizes a measurement culture. However, a potential problem with using Six Sigma by itself exists at all maturity levels through Level 4. The Six Sigma structure strives to identify improvement proposals and implement those with the most potential benefit. This may conflict with setting up organizational assets including process capability baselines. At this level, a paradox with Six Sigma exists. The classic Six Sigma project life cycle includes measuring, analyzing, improving and controlling the improved process. However, Level 4 of the CMM establishes the ability to manage a process quantitatively, thus making it predictable. Any improvement comes from fixing and stabilizing existing processes. Only at this point can a process be improved and control returned, which is CMM Level 5.

Maturity Level 5

At this level, CMM and Six Sigma work well, and both improvement projects are virtually identical. The data now exists to make an improvement investment decision as well as to predict and confirm measurable benefits. A small difference between the two improvement strategies is caused by the orientation of the CMM as an organizational change model. The process needed to deploy and institutionalize process and technology changes is more explicit in the CMM framework, which helps in creating a sustained environment for continuous improvement. Otherwise, an enterprise operating at Level 5 provides benefits similar to a best-in-class Six Sigma program.

Whereas in most cases CMM and Six Sigma work well together, some situations present conflicts for the two approaches. CMM is cited for its ability to control the process and to progress into a disciplined and managed software development environment. Six Sigma is noted for the process analysis and statistical skills that assist Black Belts when working with process problems. Motorola and Allied Signal, early adopters of Six Sigma, elected to use the CMM as a framework to start their software process improvement initiatives. They leveraged their experience with Six Sigma and were among the first enterprises to achieve CMM Level 4. Ultimately, a blend of the two approaches seems most appropriate, with Six Sigma entering at the higher maturity levels.

Source: Joseph Puffer, TeraQuest Metrics, Inc., joe.puffer@teraquest.com
Writer: Carolyn LeVasseur, Gartner Measurement

CMM^® and Capability Maturity Model^® are registered in the U.S. Patent and Trademark Office. CMM^sm Integration and CMMI^sm are service marks of Carnegie Mellon University.