9.1 Introduction 9.2 ATAM method 9.3 Example 9.4 MPM methodohar/luennot/luennot2006/Ohar9...
Transcript of 9.1 Introduction 9.2 ATAM method 9.3 Example 9.4 MPM methodohar/luennot/luennot2006/Ohar9...
1Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
9. Evaluation of software architectures9. Evaluation of software architectures
9.1 Introduction9.2 ATAM method9.3 Example9.4 MPM method9.5 Summary
2Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
9.1 Introduction9.1 Introduction
What is "architectural"?
System S
components,subsystems
not "architectural"in the context of STypically not architectural:- data structures- algorithms- details of interfaces
3Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Evaluation viewpoint to architecturesEvaluation viewpoint to architectures
Architecture description should contain the information needed to reason about the quality requirements of a system.
Architectural evaluation is usually performed against the quality requirements of a system, but alsofunctional requirements can be evaluated.
Architecture determines to what extent (most of)the quality requirements are satisfied.
4Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Quality attributes to be evaluatedQuality attributes to be evaluated
• Performance• Reliability• Availability• Security• Modifiability• Portability• Variability• Usability• Memory efficiency…
5Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Different quality attributesDifferent quality attributes
• Run-time quality attributes (e.g. performance)
• Development & evolution time quality attributes (e.g. modifiability)
6Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
What are the results of evaluation? What are the results of evaluation?
Evaluation typically yields answers to the following types of questions:
1) Is the planned architecture suitable for the system,satisfying essential quality requirements?If not, why?
2) Which of the alternative architectural solutionsis the most suitable for the system, and why?
3) How well a particular quality attribute can be achieved with the planned architecture?
Note 1: the answers are based on the architecture description, assuminga reasonable implementationNote 2: the results of the evaluation depend on the accuracy of the architecture description
7Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Why should software architectures be Why should software architectures be evaluated?evaluated?
• Architecture is the first precise description of a system• Architecture contains the most critical decisions of a system• Architectural evaluation increases the understanding of the system
• Architecture determines many process and organization aspects as well
8Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
When should software architectures be When should software architectures be evaluated?evaluated?
• After the first architectural sketch (pre-evaluation)
• After the architecture is designed,before implementation is started (full evaluation)
• After the system is implemented (sensible only inthe case of legacy systems)
9Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
How to elicit information about the system'squality properties based on architecture?
• Scenarios• Checklists• Metrics• Simulations• Experts
10Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
ScenarioScenario--based evaluation methodsbased evaluation methods
SAAM (Software Architecture Analysis Method)• concentrates on modifiability, portability, variability• developed at SEI• based on evolution-time scenarios
ATAM (Architecture Tradeoff Analysis Method)• covers all quality attributes• developed at SEI• derived from SAAM
MPM (Maintenance Prediction Method)• concentrates on maintainability (cost of maintenance)• developed by Jan Bosch• based on maintenance scenarios
11Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
9.2 ATAM (Architecture Tradeoff Analysis 9.2 ATAM (Architecture Tradeoff Analysis Method)Method)
1. Presentation
2. Analysis
3. Testing
4. Reporting
Phase 1
Phase 2
12Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Presentation (Phase 1)Presentation (Phase 1)
1. Present the ATAM method• ATAM phases• ATAM techniques (scenarios, quality attribute trees, etc.)
2. Present the business drivers• most important functions of the system• business goals of the system• constraints (economic, political etc.)
3. Present the architecture• technical constraints (operating system, hardware etc.)• external interfaces of the system• architecture description
13Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Analysis (Phase 1)Analysis (Phase 1)
4. Identify architectural solutions• identify and name used styles, patterns etc.• add explanations on how the solution achieves certain quality attributes
5. Produce quality attribute tree• refine quality requirements to the level of scenarios• prioritize by importance and difficulty
6. Analyze the architectural solutions against thescenarios
• architecture is probed against scenarios• risks, nonrisks, sensitivity points and tradeoff points identified
14Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Quality attribute treeQuality attribute tree
Quality
Performance
Modifiability
Availability
Security
quality attributes
Throughput 1000 servicerequests/sec (H,M)Authentication responsein less than 1 sec. (H,M)
Change to Web UIin 1 month (M,H)Change to Linux in 6 months (L,H)Restart after disk failurein 5 minutes (L,H)
Restart after auth servercrash in 5 minutes (M,M)
Credit card transactionsecure 99.999% (H,L)
scenarios
TransactionthroughputResponse time
Change UIChange OS
Hardware failureServer crash
Data confidentiality
refinements …
15Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
RisksRisks
Risk = potentially problematic architectural decision
example:
The rules for writing business logic modules in the second layer of your three-layer architecture arenot clearly articulated. (decision/fact in architecture)This could result in replication of functionality (rationale),thereby compromising modifiability (quality atttribute implication).
16Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
NonrisksNonrisks
Nonrisk = good decision/property in the architecture based on safe assumptions
example:
because components do not have to know the components responding to their events (rationale).
Assuming that only the actually responding componentsare registered as observers (assumption),
the use of the Observer pattern in the communication betweenthe components (decision)improves modifiability (quality attribute implication),
17Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Sensitivity pointsSensitivity points
Sensitivity point = an architectural aspect that is critical for achieving a particular quality attribute
examples:The portability of the system is sensitive to the use of theMVC model in the GUI implementation.
The modifiability of the system is sensitive to the use ofthe Abstract Factory pattern for the creation of the driverobjects.
18Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Tradeoff pointsTradeoff points
Tradeoff point = Sensitivity point that affects more than one quality attribute
examples:Use of the State pattern in the implementation of the statemachine improves modifiability but impairs performance.
Using XML for the input format improves interoperabilityof the system but impairs performance.
19Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Testing (Phase 2)Testing (Phase 2)
7. Produce testing scenarios• all stakeholders brainstorm and rank scenarios from their perspectives • voting techniques for ranking scenarios • validate & complete quality attribute tree
8. (Re)analyze the architectural solutions • highly ranked scenarios run against architecture • map highly ranked scenarios to architectural solutions & quality attributes
20Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Reporting (Phase 2)Reporting (Phase 2)
Output of ATAM
• Identification and analysis of architectural solutions• Identification of prioritized scenarios• Quality attribute tree• Identification of risks• Identification of nonrisks• Identification of sensitivity points and tradeoff points• Identification of risk themes
21Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
9.3 Example: Car service monitor system9.3 Example: Car service monitor systemA car control system needs to be extended with a subsystem that collects variouskinds of data during the running of the car, to be used for monitoring and service purposes. The control system is based on a CAN-bus. The bus is used to send messagesbetween the components in the system. The monitoring system (calledMS) listens to the messages sent along the CAN-bus. It should be possible to addlater various kinds of processing capabilities to MS, reading certain kinds of messages,performing arbitrary computation on the basis of the transferred data, possibly storing the computed data on a local database, and sending in certain cases information to a central service station through GSM. For example, MS may collect information about the usageof gears and breaks, about the speed, about engine temperatures, consumption etc.The driver should have access to the collected information through a graphical user interface and activate or passivate certain information collecting services. Since MS is not controlling critical functions, there are no hard real-timerequirements. It should be possible to receive monitored information also from externalunknown systems, and to receive monitoring requests from such external systems.The system should be easily configurable to various kinds of cars, and it should be possibleto upgrade the system at run-time.
22Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
MessageDispatcherIF
send(Msg)register(MsgType,Component)
Component
Msg
receive(Msg)
type(): MsgType
XMLMsg
GSMCompBrakeState
DBBrakeController
recordUsagecheckConditionregister(View)getState()setState()
handleEvent(Event)
BrakeViewIFupdate()
CANBus
CANFilter
MessageDispatcher
sendReport
Brake-View
BrakeModelIF
23Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Analysis (1)Analysis (1)
4. Identify architectural solutions• identify and name used styles, patterns etc.• add explanations on how the solution achieves certain quality attributes
Architectural solutionUse of the message dispatcher architectural style in the communication of components
ExplanationSupports dynamic extensibility of the system, becausenew components can be loaded at run-time,interacting with other components through messages.
24Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Analysis (2)Analysis (2)5. Produce quality attribute tree• refine quality requirements to the level of scenarios• prioritize by importance and difficulty
Quality
Modifiability
Extensibility
Performance
Availability
Usability...
Data modifiability
Functional modifiability
...
S1: Change database in1 month (H,M)S2: Change message formatin two weeks (L,H)
Change UI
Change data collecting
Component serviceresponse in 1 microsec
...
25Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Analysis (3)Analysis (3)
6. Analyze the architectural solutions against thescenarios
• architectural solutions mapped to scenarios• risks, nonrisks, sensitivity points and tradeoff points identified
Scenario: S1: Change database in 1 monthQuality attribute: ModifiabilityStimulus: Old database no more maintainedResponse: Change done in 1 month Architectural solutions: Risk Nrisk Sens TroffAbstract interface for database NP1 SP1access (interface+DB comp)
26Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
NP1: Assuming that the persistence services can be provided by a newdatabase, the use of an abstraction layer (interface + DB component)between the other components and the database makes the change ofthe underlying database easy, thus improving the modifiability of the system.
SP1: The modifiability of the system is sensitive to the use of abstractionlayer between the components and the concrete database.
27Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Testing (1)Testing (1)7. Produce testing scenarios• all stakeholders brainstorm and rank scenarios from their perspectives • voting techniques for ranking scenarios • validate & complete quality attribute tree
A scenario presented by a manager:It is possible that in the future the company will develop its own proprietaryinformation exchange format for onboard devices, instead of usingstandard CAN-bus. Therefore it should be possible to read data fromother sources than CAN-bus.
A scenario presented by a car engineer:Actually the behavior of the car is also affected by the external conditions.For example, if the temperature is low, the fuel consumption gets higher.Therefore information about external conditions from various sensors might be later added to the system.
28Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Testing (2)Testing (2)8. (Re)analyze the architectural solutions • highly ranked scenarios run against architecture • map highly ranked scenarios to architectural solutions & quality attributes
Scenario: S27: Change CAN-bus to proprietary information source in two monthsQuality attribute: ModifiabilityStimulus: CAN-bus not any more used in companyResponse: The modification done in two monthsArchitectural solutions: Risk Nrisk Sens TroffSeparate source handler NP14 SP22Abstract interface for messages NP15 SP23
29Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
9.4 MPM: Maintenance Prediction Method9.4 MPM: Maintenance Prediction Method
1. Define scenario categories2. Define scenarios3. Assign weights for scenarios4. Impact analysis5. Maintenance cost prediction
30Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Define scenario categoriesDefine scenario categories
Scenario categories • serve as the first step of finding scenarios • give structure for the scenario set• help to find a covering set of scenarios• can be based for example on enviroment change types, source of change etc.
Example:Scenario category I: New/changed processing functionalityScenario category II: New/changed hardware devicesScenario category III: New/changed infrastructureScenario category IV: New/changed implementation techniquesScenario category V: Changed UI
31Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Define scenarios (1)Define scenarios (1)
Scenario category I: New/changed processing functionalityS1: Change the way a serious damage is inferred from engine temperature data ...
Scenario category II: New/changed hardware devicesS6: Add a new sensor for the temperature of the gear box, producing CAN messages to be monitored and analyzed statistically...
Scenario category III: New/changed infrastructureS11: Change the database system to another one with similar interface,both being relational...
32Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Define scenarios (2)Define scenarios (2)
Scenario category IV: New/changed implementation techniquesS15: Change the message format from XML to binary...
Scenario category V: Changed UIS18: Add voice output for alarm situations...
33Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Assign weights for scenariosAssign weights for scenarios
Scenario Normalized probability
S1
S22
Sum 1.0
0.05
0.12
34Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Impact analysisImpact analysis
Scenario Affected components LOC
S1
...
Engine Controller component
Impact
20% 0.2x1500=300
... ... ...
S15 SourceHandlerComponentXComponentY...
... ... ... ...
30%5%5%...
0.3x1800=5400.05x200=100.05x500=25
35Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Maintenance cost predictionMaintenance cost prediction
Weighted LOC impact for a scenario:
Weight * Impact = 300*0.12 + ... = X LOC
scenario ii i
Assume average cost of LOC in the enterprise: Y € / LOC
Assume expected number of scenarios in year: Z scenarios:
=> Expected maintenance cost per year: Z * X * Y €
36Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
9.5 Summary9.5 Summary
Assessment methods provide an opportunity for such discussions that should be carried out anyway during the process
Assessment methods can be regarded as exceptionally deepand systematic architectural review techniques
Methods are not (probably deliberately) very precise, giving a lotof freedom to tailor them for a particular case & company
Methods need a significant amount of resources and commitmentwhich is not always available (especially ATAM)
37Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka
Course synthesisCourse synthesisArchitecture & software development processWell-defined & documented architecture gives theconstitution of a software system
Architecture & dependenciesArchitectural solutions reduce unnecessarydependencies in the system
Architecture & reuseProduct-line architecture and variation managementenable systematic reuse of enterprise know-how
Architecture & qualityArchitectural assessment, carried out as part of the process, ensures quality attributes