Organization structures for testing teams

Organization structures for testing teams

A goal can be described as (i) a statement of intent, or (ii) a statement of a accomplishment that an individual or an organization wants to achieve.

A goal statement relates to an area where an individual, group, or organization wants to make improvements. Goals project future states of an organization, a group, or an individual. In an organization there is often a hierarchy of goals. At the top level are general organizational goals. There are intermediate-level goals that may be associated with a particular organizational functional unit. Individual projects have specific goals. These usually reflect organizational goals. There are personal-level goals as well. Each individual in an organization has a set of goals for self-improvement so that he or she can more effectively contribute to the project, functional unit, and organization as a whole.

Goal statements can express expectations in quantitative terms or be more general in nature. For the testing goals below, goals 1 and 2 express what is to be achieved in a more quantitative manner than goals 3 and 4.

1. One-hundred percent of testing activities are planned.

2. The degree of automation for regression testing is increased from 50% to 80% over the next 3 years.

3. Testing activities are performed by a dedicated testing group.

4. Testing group members have at least a bachelor-level degree and have taken a formal course in software testing.

In general, quantitative goals are more useful. These are measurable goals, and give an organization, group, or individual the means to evaluate progress toward achieving the goal. In the testing domain, goal statements should provide a high-level vision of what testing is to accomplish in the organization with respect to quality of process and product. In addition to general testing goal statements, lower-level goal statements should be developed for all levels of testing. Goals for the education and training of testing personnel should also be included with testing goal statements. Test plans should express testing goals for each project. These reflect overall organizational testing goals as well as specific goals for the project.

The TMM itself is built on a hierarchy of high-level testing maturity goals and subgoals which support the growth of an effective software testing process and promote high software quality. TheTMMcan be used by decision-makers in an organization to develop both long- and shortterm testing goals based on the TMM goal hierarchy.

A policy can be defined as a high-level statement of principle or course of action that is used to govern a set of activities in an organization.

Because a policy provides the vision and framework for decision making, it is important to have the policy formally adopted by the organization, documented, and available for all interested parties. An intraorganizational web site is suggested as a location for policy statements. This would allow for updates and visibility within the organization. A policy statement should be formulated by a team or task force consisting of upper management, executive personnel, and technical staff. In the case of testing, a testing policy statement is used to guide the course of testing activities and test process evolution. It should be agreed upon as workable by all concerned.

Testing policy statements reflect, integrate, and support achievement of testing goals. These goals in turn often target increasing software quality and improving customer satisfaction. Test policies also provide high-level guidance as to how testing is to be done in the organization, how its effectiveness will be evaluated, who will be responsible, and what choices of resources are possible. They should be explicit enough to guide decisions on all important testing issues, for example, how to test, what to test, and who will test. Policies are not written in stone, and as an organization grows in maturity its policies will change and mature. The task force should establish documented procedures for policy change. A brief outline of a sample testing policy statement appropriate for a TMM level 2 organization follows.

Testing Policy : Organization X

Our organization, the X Corporation, realizes that testing is an important component of the software development process and has a high impact on software quality and the degree of customer satisfaction. To ensure that our testing process is effective and that our software products meet the client‘s requirements we have developed and adopted the following testing policy statement.

1.  Delivering software of the highest quality is our company goal. The presence of defects has a negative impact on software quality. Defects affect the correctness, reliability, and usability of a software product, thus rendering it unsatisfactory to the client. We define a testing activity as a set of tasks whose purpose is to reveal functional and quality- related defects in a software deliverable. Testing activities include traditional execution of the developing software, as well as reviews of the software deliverables produced at all stages of the life cycle. The aggregation of all testing activities performed in a systematic manner supported by organizational policies, procedures, and standards constitutes the testing process.

2. A set of testing standards must be available to all interested parties on an intraorganizational web site. The standards contain descriptions of all test-related documents, prescribed templates, and the methods, tools, and procedures to be used for testing. The standards must specify the types of projects that each of these items is to be associated with.

3. In our organization the following apply to all software development/ maintenance projects:

•  Execution-based tests must be performed at several levels such as unit , integration, system, and acceptance tests as appropriate for each software product.

•  Systematic approaches to test design must be employed that include application of both white and black box testing methods.

•  Reviews of all major product deliverables such as requirements and design documents, code, and test plans are required.

•  Testing must be planned for all projects. Plans must be developed for all levels of executionbased testing as well as for reviews of deliverables.

Test plan templates must be included in organizational standards documents and implemented online. A test plan for a project must be compatible with the project plan for that project. Test plans must be approved by the project manager and technical staff. Acceptance test plans must also be approved by the client.

• Testing activities must be monitored using measurements and milestones to ensure that they are proceeding according to plan.

• Testing activities must be integrated into the software life cycle and carried out in parallel with other development activities. The extended modified V-model as shown in the testing standards document has been adopted to support this goal.

• Defects uncovered during each test must be classified and recorded.

• There must be a training program to ensure that the best testing practices are employed by the testing staff.

4. Because testing is an activity that requires special training and an impartial view of the software, it must be carried out by an independent testing group. Communication lines must be established to support cooperation between testers and developers to ensure that the software is reliable, safe, and meets client requirements.

5. Testing must be supported by tools, and, test-related measurements must be collected and used to evaluate and improve the testing process and the software product.

6. Resources must be provided for continuos test process improvement.

7.     Clients/developer/tester communication is important, and clients must be involved in acceptance test planning, operational profile development, and usage testing when applicable to the project. Clients must sign off on the acceptance test plan and give approval for all changes in the acceptance test plan.

8.  A permanent committee consisting of managerial and technical staff must be appointed to be responsible for distribution and maintenance of organizational test policy statements. Whatever the nature of the test policy statement, it should have strong support and continual commitment from management. After the policy statement has been developed, approved, and distributed, a subset of the task force should be appointed to permanently oversee policy implementation and change.

Debugging Policy : Organization X

Our organization, the X Corporation, is committed to delivering highquality software to our customers. Effective testing and debugging processes are essential to support this goal. It is our policy to separate testing and debugging, and we consider them as two separate processes. Each has different psychologies, goals, and requirements. The resources, training, and tools needed are different for both. To support the separation of these two processes we have developed individual testing and debugging policies. Our debugging policy is founded on our quality goal to remove all defects from our software that impact on our customers‘ ability to use our software effectively, safely, and economically. To achieve this goal we have developed the following debugging policy statement.

1. Testing and debugging are two separate processes. Testing is the process used to detect (reveal) defects. Debugging is the process dedicated to locating the defects, repairing the code, and retesting the software. Defects are anomalies that impact on software functionality as well as on quality attributes such as performance, security, ease of use, correctness, and reliability.

2. Since debugging is a timely activity, all project schedules must allow for adequate time to make repairs, and retest the repaired software.

3.  Debugging tools, and the training necessary to use the tools, must be available to developers to support debugging activities and tasks.

4.  Developers/testers and SQA staff must define and document a set of defect classes and defect severity levels. These must be must be available to all interested parties on an intraorganizational web site, and applied to all projects.

When failures are observed during testing or in operational software they are documented. A problem, or test incident, report is completed by the developer/tester at testing time and by the users when a failure/ problem is observed in operational software. The problem report is forwarded to the development group. Both testers/developers and SQA staff must communicate and work with users to gain an understanding of the problem. A fix report must be completed by the developer when the defect is repaired and code retested. Standard problem and fix report forms must be available to all interested parties on an intraorganizational web site, and applied to all projects.

6.    All defects identified for each project must be cataloged according to class and severity level and stored as a part of the project history.

7. Measurement such as total number of defects, total number of defects/ KLOC, and time to repair a defect are saved for each project.

8.  A permanent committee consisting of managerial and technical staff must be appointed to be responsible for distribution and maintenance of organizational debugging policy statements.