Architecture Evaluation

Intro

• Design decisions, patterns are used because of the effects they have on the system.
• Hence, these choices are analysable.
• Design must be evaluated continuously during the life cycle, especially early on.
• There are many benefits to evaluation:
  • Financial.
  • Early detection of problems.
  • Captured rationale - Documented design rationale is important so that implications of modifications can be assessed.
  • Validation of requirements - opens up the requirements themselves for discussion. e.g. conflicting requirements.
  • Improved architectures.

• Use scenarios as vehicles for asking probing questions about how the arch. responds to various situations.
• Propose scenarios against a quality attribute.
• For e.g. against performance - give usage profiles that stretch the system.

SAAM

• Software Architecture Analysis Method.
• Steps:

1. Develop scenarios
2. Describe candidate architecture
3. Classify scenarios - direct (arch directly support) or indirect.
4. Perform scenario evaluations - For each indirect scenario, changes to the architecture that are necessary for it to support the scenario.
5. Reveal scenario interaction - When 2 or more indirect scenarios require change to a single component - they are said to interact in that component. Areas of high scenario interaction

can indicate poor separation of concerns in a system component.

1. Overall evaluation : Tabulation, weighting of scenarios, rating architecture support. To compare candidate architectures.

ATAM

• Architectural Tradeoff Analysis Method.
• ATAM is a refinement of SAAM.
• ATAM reveals how well an architecture satisfies particular quality goals.
• Recognises that an arch. decision tends to affect more than one quality attribute and provides insight into how quality goals trade-off against each other.

Participants in ATAM

• Evaluation team - Group is external to the project whose arch. is being evaluated. Should be competent, unbiased outsiders.
• Project decision makers - Speak on behalf of the project or have authority to make changes to it. The architect must always be included.
• Architecture stakeholders - Developers, Testers, Users, Maintainers. They are needed to articulate the specific quality attribute goals.

Goals of ATAM

• Elicit and refine a precise statement of the architecture’s driving quality attribute requirements
• Elicit and refine a precise statement of the architectural design decisions
• Evaluate the architectural design decisions to determine if they satisfactorily address the quality requirements

Outputs of ATAM

• A concise presentation of the architecture. Requirement is to be concise - so that it is understandable.
• Articulation of business goals.
• Quality requirements in terms of a collection of scenarios.
• Mapping of architectural decisions to quality requirements.
• A set of identified sensitivity and trade-off points.
  • Senstivity : Where small changes have big impact on QA's.
  • Tradeoff : Where more than 1 QA is affected.
• A set of risks and non-risks.
• A set of risk themes.

Phases of ATAM

• Partnership and preparation phase - Eval team and project decision makers.
• Evaluation phase 1 - Eval team and project decision makers.
• Evaluation phase 2 - Eval team and project decision makers and stakeholders.
• Follow up - Eval team and eval client.

Evaluation phase steps

1. Present the ATAM

• Eval leader presents the ATAM to the project team.

2. Present Business Drivers

• Project decision maker presents a system overview from a business perspective.
• Business goals, major stakeholders.
• Architectural Drivers.

3. Present Architecture

• Lead architect makes a presentation describing the arch.
• Describes patterns used, tactics, styles. Describes constraints.
• Uses views to describe the architecture.

4. Identify Architectural Approaches

• Evaluation team lists and catalogues all the patterns and approaches that are evident from the architects presentation.

5. Generate Quality Attribute Tree

• Identify, prioritize and refine the system's most important quality attributes.
  • Level 0 - Root node - utility, overall expression of how good the system is.
  • Level 1 - Quality attributes e.g. performance, modifiability, security.
  • Level 2 - Specific quality attribute refinements e.g. - Performance is divided into 1) latency and 2) transaction throughput
  • Level 3 - Scenarios. Each scenario is given a priority - Importance, Difficulty of achieving e.g. (High, High), (Medium, Low ), (Low, Low)
• The tree tells the ATAM team where to spend its time.

6. Analyse Architectural Approaches

• Examines the highest ranked scenarios one at a time.
• Probe the architectural approaches in order to identify risks, sensitivity points, and tradeoffs. Document these.
• e.g. no of DB clients affects TPS. Assignment of clients is thus a sensitivity point.

Hiatus and start of Phase 2

• Evaluation team summarizes the learning and communicates with architect.
• Questions and clarifications answered.

7. Brainstorm and prioritise scenarios

• Create and analyse scenarios that represent the various stakeholders’ interests to understand quality attribute requirements and their relative importance.
• Eval team asks stakeholders to brainstorm scenarios.
• Scenarios must be prioritized. Stakeholders vote.
• Compare scenario list from this exercise with those from the utility tree exercise.

8. Analyse Architectural Approaches

• Analyse highly ranked brainstormed scenarios from previous steps in same manner as step 6.
• i.e. identify sensitivity points, tradeoffs, risks.

9. Present Results

• All collected info from ATAM needs to be summarized and presented to stakeholders.
• Arch approaches, utility tree, discovered risks, non risks, sensitivity points and trade-offs.
• Value addition by grouping risks into risk themes and relate them to business drivers.

CBAM

• Cost Benefit Analysis Method
• ATAM misses an important consideration - the economic cost of tradeoffs.
• CBAM builds on ATAM to model the costs and benefits of architectural design decisions and is a means of optimizing such decisions.
• CBAM's goal is to maximize the difference between the benefit derived from the system and the cost of implementing the design.
• Each arch decision has a cost. For e.g. redundant HW for high avail has a cost. This can result in a benefit for the org.
• CBAM elicits these costs and benefits - so that stakeholders can decide whether to use the tactic to achieve availability. Choose arch. strategies based on their ROI - ratio of benefit to cost.

Utility

• CBAM uses a set of scenarios rather than a single scenario in ATAM. The set of scenarios are generated by varying the values of the responses.
• Every stimulus - response pair in a scenario provides some utility to the stakeholder. Utility of different possible values for the response can be compared.
• The utility-response curve depicts how the utility derived from a particular response varies as the response varies.