Wenguang Wang

Computer Science Department, University of Saskatchewan

Email: wew036@cs.usask.ca

URL: http://www.cs.usask.ca/grads/wew036

This paper is intended to describe the suggested controls and methods for producing software which meets a purchaser’s requirements. This is done primarily by preventing nonconformity at all stages from development to maintenance.

Firstly, the Framework of the quality system is described. Both the supplier’s and the purchaser’s management responsibility is defined. The supplier should establish and maintain a quality system which should be an integrated process throughout the entire life cycle.

Secondly, Life-cycle activities of the quality system are described. Contract review is used by the supplier to ensure the feasibility of the contract. The purchaser’s requirements specification should include all aspects necessary to satisfy the purchaser’s need, and be stated precisely enough by the supplier so as to allow validation during product acceptance. Development planning should define the objectives of the project, the organization of the project resources, development phases, the project schedule and the identification of related plans. Quality planning, which is a part of the development planning, should define quality objectives, input/output criteria for each development phase, identification of types of test activities, test plan, and specific responsibilities. Design and implementation are complex and should be carried out in a disciplined manner. Systematic methodologies should be applied. Testing and validation should be planned and performed during and after the implementation. Acceptance tests should be planned and performed by the purchaser to judge whether or not the product is acceptable according the contract when the supplier is ready to deliver the validated product. Issues about the replication, delivery and installation should be stated after the product is ready for initial installation. Maintenance procedures should be established and maintained by the supplier after the initial delivery and installation. It should be planned, and be supported by maintenance organization. Maintenance records and reports should also be retained.

Thirdly, Supporting activities of the quality system are described. Configuration management provides a mechanism for controlling the versions of each software item. Document control is procedures to control all documents that related to this part of ISO 9000. Quality records should be maintained to demonstrate achievement of the required quality and the effective operation of the quality system. Measurement includes product measurement and process measurement. These metrics should be reported and used to manage the development and delivery process and the software product. The supplier should provide rules, practices and conventions in order to make the quality system specified in this part of ISO 9000 effective. Tools and techniques should be used to make the quality system guidelines effective. The supplier should ensure that a purchased product or service conforms to specified requirements. The supplier should establish and maintain procedures for validation, storage, protection and maintenance of included software product. The supplier should establish and maintain procedures for identifying the training needs and provide training.

ISO 9000-3 covers a broad area of software projects. However, it is too brief, especially in the design and implementation stages. It only describes what should be done, but no instructions of how to do. Based on this standard, different suppliers may get very different results because they have different understanding about the quality system.

In the life-cycle activities of the quality system, ISO 9000-3 only describes each activity separately. The relationships among the activities are not described. The sequence of the activities looks like the waterfall model which is not suitable to model complex design and implementation.

How to manage the change of the development process is important in complex projects. However, ISO 9000-3 does not describe how to manage the change of the development process.

This is an abridged version of the Capability Maturity Model (CMM) which is a software quality and management standard developed by Software Engineering Institute. There are five levels in CMM. Level 1 is initial or ad hoc level. Key practices from level 2 to level 5 are described briefly.

Level 2 is the repeatable level, and focuses on the project management. It includes following practices. The Requirements Management is to establish and maintain an agreement with the customer on the requirements for the software project. The Software Project Planning is to establish reasonable plans for performing the software engineering and for managing the project. The Software Project Tracking and Oversight is to let the actual progress of the project comply the project plan defined in the Software Project Planning key process area. The Software Subcontract Management is to select qualified software subcontractors and manage them effectively. The Software Quality Assurance is to review and audit the software products and activities to verify that they comply the applicable procedures and standards and provide these reviews and audits to appropriate manages. The Software Configuration Management is to establish and maintain the integrity of the products of the software project throughout the project life cycle.

Level 3 is the defined level, and focuses on engineering process. It includes following practices. The Organization Process Focus is to define and maintain the organization’s standard software process, along with related process assets. The Training Program is to develop the skills and knowledge of individuals so they can perform their roles effectively and efficiently. The Integrated Software Management is to integrate the software engineering and management activities into a coherent, defined software process. The Software Product Engineering is to perform a well-defined engineering process that integrates all the software engineering activities to produce good software products. The Intergroup Coordination is to establish a means for the software engineering group to participate actively with the other engineering groups in order to satisfy the customer’s needs better. The Peer Reviews is to establish and maintain a meaning for identifying defects of the software work products by the producers’ peers.

Level 4 is the managed level, and focuses on product and process quality. It includes following practices. The Quantitative Process Management is to control the process performance of the software project quantitatively. The Software Quality Management is to develop a quantitative understanding of the quality of the project’s software products and achieve specific quality goals.

Level 5 is the optimizing level, and focuses on continuous process improvement. It includes following practices. The Defect Prevention is to identify the cause of defects and prevent them from recurring. The Technology Change Management is to identify new technologies and track them into the organization in an orderly manner. The Process Change Management is to continually improve the software processes used in the organization with the intent of improving software quality, increasing productivity, and decreasing the cycle time for product development.

It is a good idea that SEI divides the CMM to different levels. These levels can be used to identify how much a corporation has achieved in quality and management process. When adopting the CMM, a corporation can also achieve these levels step by step.

However, there is no analysis in CMM about the characteristics of each level. Different level has different advantages and overheads. Maybe the overhead of some levels is bigger than the gains for some corporations.

This paper compares the ISO 9001 with the CMM (Capability Maturity Model). The CMM and the ISO 9000 series of standards have the common concern of quality and process management. To compare them, 20 clauses of ISO 9001 are mapped to CMM key practices in this paper.

Although some issues in ISO 9001 are not covered in the CMM, and vice versa, there is a strong correlation between them. The biggest difference between the two documents is the explicit emphasis of the CMM on continuous process improvement. Another difference is that the CMM focuses strictly on software, while ISO 9001 has a much broader scope. The biggest similarity between the two documents is their bottom line: document what you will do; do what you have documented.

Because the ISO 9001’s high level of abstraction, auditors may interpret it in very different ways. Every key process area at level 2 of the CMM are strongly related to ISO 9001, and every key process area of the CMM is at least weakly related to ISO9001 under some interpretation. However, a level 1 organization can receive ISO 9001 certificates. Because the CMM provides more detailed guidance and software specificity, it may be a better choice for an organization. After it reaches level 2 or higher, it should be relatively straightforward to obtain the ISO 9001 certification. In any case, organizations should focus on the improvement to build a competitive advantage, not on achieving a score.

The author compares the CMM and ISO 9001 in many details which show some insight of the standards. However, it does not discuss the effort needed for auditors to master the standards. Because the auditors are very important to perform the standards, this topic is not trivial.

The SPICE is an International Standard which provides a framework for the assessment of software processes. This standard can satisfy the needs of acquirers, suppliers and assessors, and their individual requirements from within a single source. This standard focuses on the process assessment and the use of the process assessment. It consists of 9 parts.

Part 1 is an introduction. Part 2 defines the fundamental practices that are essential to software engineering. These practices are categorized into five categories: Customer-Supplier process, Engineering process, Project process, Support process, and Organization process. Six capability levels can express the evolving of the process capability. An assessment is carried out by assessing selected processes against the process model defined in this part.

Part 3 to 6 are all about the process assessment. Part 3 defines a framework for conducting an assessment, and sets out the basis for rating, scoring and profiling process capabilities. Part 4 provides guidance on the conduct of team-based software process assessments. Part 5 defines the framework elements required to construct an instrument to assist an assessor in the performance of an assessment. Part 6 describes the competence, education, training and experience of assessors that are relevant to conducting process assessments.

Part 7 describes how to define the inputs to and use the results of an assessment for the purposes of process improvement. A process improvement methodology which is an eight step model for improving software processes within a continuous improvement cycle is described. Cultural issues and the management of software process improvement are also covered. Part 8 describes how to define the inputs to and use the results of an assessment for the purpose of process capability determination. A main reason for carrying out a process capability determination is to obtain information upon which to base a procurement-related decision. There are two alternative approaches to process capability determination: the core process capability determination is a minimum, streamlined set of activities without any partners or sub-contractors being involved. The extended process capability determination is applicable when an enhanced capability is proposed, or when consortia or sub-contractors are involved.

Part 9 is a consolidated vocabulary of all terms specifically defined for the purposes of this International Standard.

AlThough the part 2 of the SPICE describes several levels and categories of process models, the whole standard focuses on the process assessment. Part 3 and part 4 are two central parts of the SPICE. They describe the framework and guidance of the process assessment. Part 5 and part 6 provide the instruments, tools and assessors for the process assessment. The assessment is based on the process model defined in part 2. Part 7 and part 8 are the proposes of the process assessment: process improvement or process capability determination.

The CMM and ISO 9001 are mainly about the process management. They are common in that they must establish fairly definite requirements or contracts before further software development. Therefore, it is not very suitable for commercial software corporations, which have a large number of customers and have no clear requirements before the development.

There is a common point in the CMM and ISO 9001, which is a three-stage activity: planning before implementation, following the plan in the implementation, and recording the result after the implementation. Even if the plan should be change, a change plan is also needed. The use of the plan and result documentation can make the development in a disciplined way. Following the plan document during the implementation can decrease subject factors and make the process repeatable. The recorded result can be used to analyze and improve the process. There is a tradeoff between a strict plan and a flexible plan. A strict plan makes the implementation in an orderly manner, but impedes the change during the implementation in some extent. A flexible plan makes it easy to be changed and improved, but may make the implementation disorderly.

The intention of the CMM and the ISO 9001 is to help organizations improve the process and quality management. Therefore, the organization should focus on how to improve the management level, not on getting a score. If the organization only wants to get the certificate of the two standards, it may not grasp the pith of the standards. Moreover, after getting the certification, the organization may not continue to perform the standards.

The SPICE focuses on the assessment of the process. It provides more details about the assessment than the CMM and ISO 9001. The part 2 of the SPICE describes a process model with several categories and levels which is similar to the CMM. Because the CMM contains more details of the process model than the part 2 of the SPICE, we can expect to get better result if we apply the assessment principles of the SPICE to the key areas of the CMM.

A good standard is important for the organization to improve the quality and process management. There is another crucial factor for performing the standard effectively -- the ability of the persons who audit and conduct the standard. Every standard must be performed by persons. The ability of the persons can affect the effect of the standards greatly. For example, ISO 9001 certification can be obtained by a level 0 (CMM) organization. However, if the auditors can interpret the ISO 9001 standard in a better way, the organization which has the ISO 9001 certification can achieve the level 2 or level 3 of the CMM. Due to the great importance of the persons, topics about how to train the persons should be studied.

[1] Quality management and quality assurance standards Part 3: Guidelines for the application of ISO9001 to the development, supply and maintenance of software. ISO 9000-3:1991(E)

[2] Appendix C: Abridged Version of the Key Practices for Capability Maturity Model (CMM). Http://rbse.jsc.nasa.gov/process_maturity/CMM/TR25/tr25_xc.html

[3] Mark C. Paulk. How ISO 9001 Compares With The CMM. IEEE Software, January 1995.

[4] SPICE: Software Process Improvement and Capability dEtermination.