Towards the Evaluation of Systems of Systems Architecture Daniel S. Santos Brauner R. N. Oliveira Milena Guessi Flavio Oquendo Marcio Delamaro Elisa Yumi Nakagawa
Towards the Evaluation of Systems of Systems Architecture
Daniel S. Santos Brauner R. N. Oliveira
Milena Guessi Flavio Oquendo
Marcio Delamaro Elisa Yumi Nakagawa
Schedule
• Introduction • Systems of Systems
• Architecture Evaluation Methods
• Evaluation of SoS Software Architectures • Evaluation Methods
• Quality attributes
• Challenges
• Conclusion
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
2
Introduction
Systems of Systems "A set or arrangement of independent, useful systems integrated into
larger systems that deliver unique capabilities " (DoD, 2008).
Challenges: • Constituents systems are often developed and
maintained by different organizations. • These organizations can have their own stakeholders,
development teams, and processes. • Evaluation of SoS software architectures could ensure
that quality attributes are satisfied
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
3
Introduction
Architecture Evaluation • Can be used to compare and identify strengths and weaknesses of
different architectural alternatives
• Guide the maintenance or indicate new opportunities for enhancing software architectures
• It is essential for ensuring that software architectures meet desired quality attributes
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
4
Introduction
• Goals: To present the state of the art on SoS software architecture evaluation as well as new research perspectives.
• Based on results of a Systematic Literature Review (SLR)1. • Overall, 16 primary studies were included in this SLR.
1 Available at http://goo.gl/PU12iQ (last accessed on 07/13/2014)
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
5
Primary studies selected
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
6
Evaluation of SoS Software Architecture
Evaluation is performed in: • Design phase
• Comparing and identifying strengths and weaknesses of different architectural alternatives.
• Maintenance phase • Intending to analyze the flexibility and ability to evolution
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
7
Evaluation Methods
• No convergence in using a specific type of evaluation method.
• Architecture Trade-off Analysis Method (ATAM) is the most popular.
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
8
Frequency of use of methods and techniques
Maturity of Evaluation Methods
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
9
31%
13% 31%
19%
6% Demonstration
Expert Opinion
Case Study
Application inIndustry
No Assessment
Maturity of the approaches
Quality Attributes
Evaluation methods can either focus on single or several quality attributes.
• Several Quality Attributes • Scenario-based methods
• No focus on specific quality attributes
• Few tangible quality attributes • simulation-based and mathematical modeling methods
• Reliability, performance, complexity, and flexibility.
• None of the studies discuss the use of quality models during architectural evaluation.
Towards the Evaluation of Systems of Systems Architecutre
CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
10
Quality Attributes
Recommendations • Evaluation methods for SoS should take into account several quality attributes.
• Evaluation methods should be able of measuring and classifying quality attributes.
• Quality models would be relevant for evaluating SoS architectures • Provide standardization for quality attributes
• Guide the identification and prioritization of quality attributes
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
11
Conclusion
• There is still no consensus on what exactly should be considered during evaluation.
• There is still several difficulties and challenges
• Appropriate and scalable evaluation approaches still need to be developed.
Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –
WDES 2014
12