Quality Attribute Workshop(QAW) Method

QAW Method

The QAW is a facilitated, early intervention method used to generate, prioritize, and refine quality attribute scenarios before the software architecture is completed. The QAW is focused on system-level concerns and specifically the role that software will play in the system. The QAW is dependent on the participation of system stakeholders—individuals on whom the system has significant impact, such as end users, installers, administrators (of database management systems [DBMS], networks, help desks, etc.), trainers, architects, acquirers, system and software engineers, and others. The group of stakeholders present during any one QAW should number at least 5 and no more than 30 for a single workshop. In preparation for the workshop, stakeholders receive a “participants handbook” providing example quality attribute taxonomies, questions, and scenarios. If time allows, the handbook should be customized to the domain of the system and contain the quality attributes, questions, and scenarios that are appropriate to the domain and the level of architectural detail available.

The contribution of each stakeholder is essential during a QAW; all participants are expected to be fully engaged and present throughout the workshop. Participants are encouraged to comment and ask questions at any time during the workshop. However, it is important to recognize that facilitators may occasionally have to cut discussions short in the interest of time or when it is clear that the discussion is not focused on the required QAW outcomes. The QAW is an intense and demanding activity. It is very important that all participants stay focused, are on time, and limit side discussions throughout the day.

The QAW involves the following steps:

1.QAW Presentation and Introductions

2.Business/Mission Presentation

3.Architectural Plan Presentation

4.Identification of Architectural Drivers

5.Scenario Brainstorming

6.Scenario Consolidation

7.Scenario Prioritization

8.Scenario Refinement

The following sections describe each step of the QAW in detail.

Step 1: QAW Presentation and Introductions

In this step, QAW facilitators describe the motivation for the QAW and explain each step of the method. We recommend using a standard slide presentation that can be customized depending on the needs of the sponsor. Next, the facilitators introduce themselves and the stakeholders do likewise, briefly stating their background, their role in the organization, and their relationship to the system being built.

Step 2: Business/Mission Presentation

After Step 1, a representative of the stakeholder community presents the business and/or mission drivers for the system. The term “business and/or mission drivers” is used carefully here.

Some organizations are clearly motivated by business concerns such as profitability, while others, such as governmental organizations, are motivated by mission concerns and find profitability meaningless. The stakeholder representing the business and/or mission concerns (typically a manager or management representative) spends about one hour presenting the system’s business/mission context

• high-level functional requirements, constraints, and quality attribute requirements

During the presentation, the facilitators listen carefully and capture any relevant information that may shed light on the quality attribute drivers. The quality attributes that will be refined in later steps will be derived largely from the business/mission needs presented in this step.

Step 3: Architectural Plan Presentation

While a detailed system architecture might not exist, it is possible that high-level system descriptions, context drawings, or other artifacts have been created that describe some of the system’s technical details. At this point in the workshop, a technical stakeholder will present the system architectural plans as they stand with respect to these early documents. Information in this presentation may include

• plans and strategies for how key business/mission requirements will be satisfied

• key technical requirements and constraints—such as mandated operating systems, hardware, middleware, and standards—that will drive architectural decisions

• presentation of existing context diagrams, high-level system diagrams, and other written Descriptions

Step 4: Identification of Architectural Drivers

During steps 2 and 3, the facilitators capture information regarding architectural drivers that are key to realizing quality attribute goals in the system. These drivers often include high-level requirements, business/mission concerns, goals and objectives, and various quality attributes. Before undertaking this step, the facilitators should excuse the group for a 15-minute break, during which they will caucus to compare and consolidate notes taken during steps 2 and 3. When the stakeholders reconvene, the facilitators will share their list of key architectural drivers and ask the stakeholders for clarifications, additions, deletions, and corrections. The idea is to reach a consensus on a distilled list of architectural drivers that include high-level requirements, business drivers, constraints, and quality attributes. The final list of architectural drivers will help focus the stakeholders during scenario brainstorming to ensure that these concerns are represented by the scenarios collected.

Step 4: Identification of Architectural Drivers

During steps 2 and 3, the facilitators capture information regarding architectural drivers that are key to realizing quality attribute goals in the system. These drivers often include high-level requirements, business/mission concerns, goals and objectives, and various quality attributes. Before undertaking this step, the facilitators should excuse the group for a 15-minute break, during which they will caucus to compare and consolidate notes taken during steps 2 and 3. When the stakeholders reconvene, the facilitators will share their list of key architectural drivers and ask the stakeholders for clarifications, additions, deletions, and corrections. The idea is to reach a consensus on a distilled list of architectural drivers that include high-level requirements, business drivers, constraints, and quality attributes. The final list of architectural drivers will help focus the stakeholders during scenario brainstorming to ensure that these concerns are represented by the scenarios collected.

Step 5: Scenario Brainstorming

After the architectural drivers have been identified, the facilitators initiate the brainstorming process in which stakeholders generate scenarios. The facilitators review the parts of a good scenario (stimulus, environment, and response) and ensure that each scenario is well formed during the workshop.

Each stakeholder expresses a scenario representing his or her concerns with respect to the system in round-robin fashion. During a nominal QAW, at least two round-robin passes are made so that each stakeholder can contribute at least two scenarios. The facilitators ensure that at least one representative scenario exists for each architectural driver listed in Step 4. Scenario generation is a key step in the QAW method and must be carried out with care. We suggest the following guidance to help QAW facilitators during this step:

1. Facilitators should help stakeholders create well-formed scenarios. It is tempting for stakeholders to recite requirements such as “The system shall produce reports for users.”

While this is an important requirement, facilitators need to ensure that the quality attribute aspects of this requirement are explored further. For example, the following scenario sheds more light on the performance aspect of this requirement: “A remote user requests a database report via the Web during peak usage and receives the report within five seconds. ”

Note that the initial requirement hasn’t been lost, but the scenario further explores the performance aspect of this requirement. Facilitators should note that quality attribute names by themselves are not enough. Rather than say “the system shall be modifiable,” the scenario should describe what it means to be modifiable by providing a specific example of a modification to the system vis-à-vis a scenario.

2. The vocabulary used to describe quality attributes varies widely. Heated debates often revolve around to which quality attribute a particular system property belongs. It doesn’t matter what we call a particular quality attribute, as long as there’s a scenario that describes what it means.

3.Facilitators need to remember that there are three general types of scenarios and to ensure that each type is covered during the QAW:

a. use case scenarios - involving anticipated uses of the system b. growth scenarios - involving anticipated changes to the system

c. exploratory scenarios - involving unanticipated stresses to the system that can include uses and/or changes

4.Facilitators should refer to the list of architectural drivers generated in Step 4 from time to time during scenario brainstorming to ensure that representative scenarios exist for each one.

Step 6: Scenario Consolidation

After the scenario brainstorming, similar scenarios are consolidated when reasonable.

To do that, facilitators ask stakeholders to identify those scenarios that are very similar in content. Scenarios that are similar are merged, as long as the people who proposed them agree and feels that their scenarios will not be diluted in the process. Consolidation is an important step because it helps to prevent a “dilution” of votes during the prioritization of scenarios (Step 7).

 Such a dilution occurs when stakeholders split their votes between two very similar scenarios. As a result, neither scenario rises to importance and is therefore never refined (Step 8). However, if the two scenarios are similar enough to be merged into one, the votes might be concentrated, and the merged scenario may then rise to the appropriate level of importance and be refined further.

Facilitators should make every attempt to reach a majority consensus with the stakeholders before merging scenarios. Though stakeholders may be tempted to merge scenarios with abandon, they should not do so. In actuality, very few scenarios are merged.

Step 7: Scenario Prioritization

Prioritization of the scenarios is accomplished by allocating each stakeholder a number of votes equal to 30% of the total number of scenarios generated after consolidation. The actual number of votes allocated to stakeholders is rounded to an even number of votes at the discretion of the facilitators. For example, if 30 scenarios were generated, each stakeholder gets 30 x

0.3, or 9, votes rounded up to 10. Voting is done in round-robin fashion, in two passes. During each pass, stakeholders allocate half of their votes. Stakeholders can allocate any number of their votes to any scenario or combination of scenarios. The votes are counted, and the scenarios are prioritized accordingly.

Step 8: Scenario Refinement

After the prioritization, depending on the amount of time remaining, the top four or five scenarios are refined in more detail. Facilitators further elaborate each one, documenting the following:

• Further clarify the scenario by clearly describing the following six things:

1.stimulus - the condition that affects the system

2.response - the activity that results from the stimulus

3.source of stimulus - the entity that generated the stimulus

4.environment - the condition under which the stimulus occurred

5.artifact stimulated - the artifact that was stimulated

6.response measure - the measure by which the system’s response will be evaluated

• Describe the business/mission goals that are affected by the scenario.

• Describe the relevant quality attributes associated with the scenario.

• Allow the stakeholders to pose questions and raise any issues regarding the scenario. Such questions should concentrate on the quality attribute aspects of the scenario and any concerns that the stakeholders might have in achieving the response called for in the scenario.

See the example template for scenario refinement in Appendix A. This step continues until time runs out or the highest priority scenarios have been refined. Typically, time runs out first.