Architecture Evaluation

From Suhrid.net Wiki
Jump to navigationJump to search

Intro

  • Design decisions, patterns are used because of the effects they have on the system.
  • Hence, these choices are analysable.
  • Design must be evaluated continuously during the life cycle, especially early on.
  • There are many benefits to evaluation:
    • Financial.
    • Early detection of problems.
    • Captured rationale - Documented design rationale is important so that implications of modifications can be assessed.
    • Validation of requirements - opens up the requirements themselves for discussion. e.g. conflicting requirements.
    • Improved architectures.
  • Use scenarios as vehicles for asking probing questions about how the arch. responds to various situations.
  • Propose scenarios against a quality attribute.
  • For e.g. against performance - give usage profiles that stretch the system.

SAAM

  • Software Architecture Analysis Method.
  • Steps:
  1. Develop scenarios
  2. Describe candidate architecture
  3. Classify scenarios - direct (arch directly support) or indirect.
  4. Perform scenario evaluations - For each indirect scenario, changes to the architecture that are necessary for it to support the scenario.
  5. Reveal scenario interaction - When 2 or more indirect scenarios require change to a single component - they are said to interact in that component. Areas of high scenario interaction

can indicate poor separation of concerns in a system component.

  1. Overall evaluation : Tabulation, weighting of scenarios, rating architecture support. To compare candidate architectures.

ATAM

  • Architectural Tradeoff Analysis Method.
  • ATAM is a refinement of SAAM.
  • ATAM reveals how well an architecture satisfies particular quality goals.
  • Recognises that an arch. decision tends to affect more than one quality attribute and provides insight into how quality goals trade-off against each other.

Participants in ATAM

  • Evaluation team - Group is external to the project whose arch. is being evaluated. Should be competent, unbiased outsiders.
  • Project decision makers - Speak on behalf of the project or have authority to make changes to it. The architect must always be included.
  • Architecture stakeholders - Developers, Testers, Users, Maintainers. They are needed to articulate the specific quality attribute goals.

Goals of ATAM

  • Elicit and refine a precise statement of the architecture’s driving quality attribute requirements
  • Elicit and refine a precise statement of the architectural design decisions
  • Evaluate the architectural design decisions to determine if they satisfactorily address the quality requirements

Outputs of ATAM

  • A concise presentation of the architecture. Requirement is to be concise - so that it is understandable.
  • Articulation of business goals.
  • Quality requirements in terms of a collection of scenarios.
  • Mapping of architectural decisions to quality requirements.
  • A set of identified sensitivity and trade-off points.
    • Senstivity : Where small changes have big impact on QA's.
    • Tradeoff : Where more than 1 QA is affected.
  • A set of risks and non-risks.
  • A set of risk themes.

Phases of ATAM

  • Partnership and preparation phase - Eval team and project decision makers.
  • Evaluation phase 1 - Eval team and project decision makers.
  • Evaluation phase 2 - Eval team and project decision makers and stakeholders.
  • Follow up - Eval team and eval client.

Evaluation phase steps

1. Present the ATAM

  • Eval leader presents the ATAM to the project team.

2. Present Business Drivers

  • Project decision maker presents a system overview from a business perspective.
  • Business goals, major stakeholders.
  • Architectural Drivers.

3. Present Architecture

  • Lead architect makes a presentation describing the arch.
  • Describes patterns used, tactics, styles. Describes constraints.
  • Uses views to describe the architecture.

4. Identify Architectural Approaches

  • Evaluation team lists and catalogues all the patterns and approaches that are evident from the architects presentation.

5. Generate Quality Attribute Tree

  • Identify, prioritize and refine the system's most important quality attributes.
    • Level 0 - Root node - utility, overall expression of how good the system is.
    • Level 1 - Quality attributes e.g. performance, modifiability, security.
    • Level 2 - Specific quality attribute refinements e.g. - Performance is divided into 1) latency and 2) transaction throughput
    • Level 3 - Scenarios. Each scenario is given a priority - Importance, Difficulty of achieving e.g. (High, High), (Medium, Low ), (Low, Low)
  • The tree tells the ATAM team where to spend its time.

6. Analyse Architectural Approaches

  • Examines the highest ranked scenarios one at a time.
  • Probe the architectural approaches in order to identify risks, sensitivity points, and tradeoffs. Document these.
  • e.g. no of DB clients affects TPS. Assignment of clients is thus a sensitivity point.

Hiatus and start of Phase 2

  • Evaluation team summarizes the learning and communicates with architect.
  • Questions and clarifications answered.

7. Brainstorm and prioritise scenarios

  • Create and analyse scenarios that represent the various stakeholders’ interests to understand quality attribute requirements and their relative importance.
  • Eval team asks stakeholders to brainstorm scenarios.
  • Scenarios must be prioritized. Stakeholders vote.
  • Compare scenario list from this exercise with those from the utility tree exercise.

8. Analyse Architectural Approaches

  • Analyse highly ranked brainstormed scenarios from previous steps in same manner as step 6.
  • i.e. identify sensitivity points, tradeoffs, risks.

9. Present Results

  • All collected info from ATAM needs to be summarized and presented to stakeholders.
  • Arch approaches, utility tree, discovered risks, non risks, sensitivity points and trade-offs.
  • Value addition by grouping risks into risk themes and relate them to business drivers.

CBAM

  • Cost Benefit Analysis Method
  • ATAM misses an important consideration - the economic cost of tradeoffs.
  • CBAM builds on ATAM to model the costs and benefits of architectural design decisions and is a means of optimizing such decisions.
  • CBAM's goal is to maximize the difference between the benefit derived from the system and the cost of implementing the design.