SoftwareReconstruction - Chapter 20 - Software Architecture in Practice

Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 1

L. Bass, P. Clements, R. Kazman: „Software Architecture in Practice“

Chapter 20: Architecture Reconstruction and Conformance

Maciej Lukaszewski


1. Introduction

2. Architecture Reconstruction Process.

2.1. Raw view extraction

2.2. Database Construction

2.3. View Fusion

2.4. Finding Violations.

3. Guidelines

4. Summary

Agenda


Introduction: why reconstruction?

What if system already exists?

Documentation lost?

Documentation no longer synchronized?

The answer is: reverse engineering!


Architecture reconstruction:

„...techniques that allow an analyst to build, maintain, and understand a representation of an existing architecture...“

Two purposes:

- to document an architecture

- to ensure conformance between as-built and as-designed architecture

Introduction II


Reconstruction, but how?

- using automated and semiautomated extraction tools

- probing the original design intent of the architect

→ combination of both techniques

→ in practise: reconstruction not fully automated

Introduction II


First: workbench!

Phases:

1. Raw view extraction

2. Database construction

3. View fusion and manipulation.

4. Architecture analysis: finding violations

Architecture Reconstruction Process


„...analyzing a system's existing design and implementation artifacts to construct one or more models of it.“

Goals:

- extracting a target set of architectural views to support documentation effort

- answers on „what components are affected if I rewrite component A?“, „How to refactor layering to remove cyclic dependencies“

→ elements of interest (files, functions, variables) and their relationships come from source artifacts (code, header files, build files)

Raw view extraction


Source element

Relation Target element

Description

File includes File C preprocessor #include of one file by another

file contains Function Definition of a funcion in a file

A typical list of the elements and several relationships among them.

Raw view extraction II


Two categories of information obtained:

- static – observing only system artifacts

- dynamic – observing how the system runs

Goal: fusing both to create more accurate system views

Raw view extraction III


What if system changes at runtime?

→ capturing runtime configuration

Why capturing?

- dynamic elements

- late binding

- static reverse engineering not possible (peer-2-peer, SOA, cloud)

Raw view extraction IV


Last, but not least → reduce irrelevant information

Example: operating-system specific libraries

Raw view extraction V


- storing extracted information in database (large amount of data, manipulations of data error-prone)

- when using a workbench (set of tools) choosing a database and deciding on internal representations of views required

- often extracted data from the code too complex → white noise

Database construction


• Raw views may be too specific to aid in architectural understanding.

Accurate, but not helpful

Database construction II


How to filter information?

Manipulating views (e.g. hiding methods inside class definitions)

Showing abstractions (e.g. identifing distinct layers)

Database construction III


Views which have been extracted are manipulated to create fused views

e.g. static call view fused with dynamic call view to provide more accurate information about the part of system

Creating a fused view = creating a hypothesis about the architecture and a visualization of it

- views can be interpreted, further refined, or rejected

- no universal completion criteria for this process

View fusion


● Tool for static code review written in Java, support of other programming languages planed

● „Bridging the gap between the architects and developers“

● More http://www.hello2morrow.com/products/sonarj

Short excursion: SonarJ (Sonargraph for Java)



http://www.hello2morrow.com/products/sonarj

http://www.hello2morrow.com/products/sonarj


SonarJ extracts facts from a set of source code files and lets you define a set of layers and vertical slices through those layers.

View fusion: example of a fused view


View fusion phase gave us a lot of hypotheses– those inform of architectural elements (layers),

constraints, relationships

– Hypotheses need to be tested in order to check conformance with architect's intention

– Two major possibilities for maintaining conformance between code and architecture:

- conformance by construction

- conformance by analysis

Architecture analysis: finding violations


Previous figure with added relationships and constraints, which have been added by an architect to reflect design intent.

Architecture analysis II: example


Black Arc: SonarJ has found a violation. The arc between the

Service layer and JUnit.

Architecture analysis III


● Have a goal and a set of objectives or questions in mind before undertaking an architecture reconstruction project.

● Obtain some representation of the system before beginning the detailed reconstruction process.

● Identifying layers is a good place to start● In some cases: disregard the existing documentation● Tools can not do all the work! Involvement of people familiar

with the system required.

Guidelines


● Architecture reconstruction ensures that a system is built the way it was designed and that it evolves in a way that is consistent with developer's intentions.

● Reconstruction needed when:

- documentation lost or out-of-date

- checking conformance

- starting the reengineering process to a new desired architecture

Summary


● The software architecture reconstruction process comprises the following phases:

- raw view extraction

- database construction

- view fusion

- architecture analysis

Summary II


1. What inferences can you make about a system's module structures from examining a set of behavioral traces gathered dynamically?

Discussion


2. Suppose you hypothesize that a system makes use of particular tactics to achieve a particular quality attribute. Fill in the columns of the table below to show how you would go about verifying your hypothesis.

Discussion

Tactics for.. Interviewing experts on the system

Analyzing structure of source code files

Static analysis of source code

Dynamic analysis of system's execution

Availability

Interoperability

Modifiability

Performance

Security

Testability

Usability


1. Introduction

2. Architecture Reconstruction Process.

2.1. Raw view extraction

2.2. Database Construction

2.3. View Fusion

2.4. Finding Violations.

3. Guidelines

4. Summary

Agenda


Introduction: why reconstruction?

What if system already exists?

Documentation lost?

Documentation no longer synchronized?

The answer is: reverse engineering!

Imaginary situation: → so far discussed architectural decisions to achieve goals

and requirements, but: → you come to a company with some complex solutions

already working and the architecture not recorded by actual developers?

→ system already exists, but no documentation describing original processes

→ documentation available, but no longer up-to-date due to series of changes

How to maintain sucha system and test quiality attributes?


Architecture reconstruction:

„...techniques that allow an analyst to build, maintain, and understand a representation of an existing architecture...“

Two purposes:

- to document an architecture

- to ensure conformance between as-built and as-designed architecture

Introduction II

In architecture reconstruction, the as-built architecture of an implemented system is reverse engineered from existing system artifacts

- the opposite to forward engineering: elements mapped to specific implementation elements (functions, classes, files, objects)

- typically its a very tool-intensive activity


Reconstruction, but how?

- using automated and semiautomated extraction tools

- probing the original design intent of the architect

→ combination of both techniques

→ in practise: reconstruction not fully automated

Introduction II

In practise: not fully automatic, rather an interpretive, interactive and iterative process involving many

activities and skills of experts and architects


First: workbench!

Phases:

1. Raw view extraction

2. Database construction

3. View fusion and manipulation.

4. Architecture analysis: finding violations

Architecture Reconstruction Process

→ requires skillfull application of tools, not possible with a single tool , e.x. implementations written in different scripting languages, no tool speaks every language

→ workbench: tool set – should be open (easy to integrate new tools), and provide an integration framework whereby new tools dont impact existing data and tools


„...analyzing a system's existing design and implementation artifacts to construct one or more models of it.“

Goals:

- extracting a target set of architectural views to support documentation effort

- answers on „what components are affected if I rewrite component A?“, „How to refactor layering to remove cyclic dependencies“

→ elements of interest (files, functions, variables) and their relationships come from source artifacts (code, header files, build files)

Raw view extraction


Source element

Relation Target element

Description

File includes File C preprocessor #include of one file by another

file contains Function Definition of a funcion in a file

A typical list of the elements and several relationships among them.

Raw view extraction II

→ Typical list of the elements and relationships that might be extracted

→ each relationship gives different information about the system

1. calls – helps to build a call graph2. includes – set of dependencies3. access_read access_write – how data is usedEtc.


Two categories of information obtained:

- static – observing only system artifacts

- dynamic – observing how the system runs

Goal: fusing both to create more accurate system views

Raw view extraction III


What if system changes at runtime?

→ capturing runtime configuration

Why capturing?

- dynamic elements

- late binding

- static reverse engineering not possible (peer-2-peer, SOA, cloud)

Raw view extraction IV

So, What is this „dynamic“-thing all about?

Why capturing dynamic elements?

- dynamic elements- It may be possible that system changes at runtime. In that case the runtime configuration should be captured and used when carrying out the reconstruction. Example: configuration file read in at startup and connections are made or elements executed.

- late binding – mechanism where a method being called upon an object is looked up by name at runtime ex. Polymorphism, plug-ins, pointers.

- peer-2-peer, SOA and cloud computing only dynamically, because precise topology of a system may not be determined until runtime. It depends on availability, loading , so cannot be reverse engineered using static tools. Therefore tools that generate dynamic info about the system.


Last, but not least → reduce irrelevant information

Example: operating-system specific libraries

Raw view extraction V

Before loading a raw view into the database → remove irrelevant information

For example: while building or using libraries that are OS-specific, you may be only interested in a specific OS (e.g. Linux) and discard the libraries for other platforms.


- storing extracted information in database (large amount of data, manipulations of data error-prone)

- when using a workbench (set of tools) choosing a database and deciding on internal representations of views required

- often extracted data from the code too complex → white noise

Database construction


• Raw views may be too specific to aid in architectural understanding.

Accurate, but not helpful

Database construction II

White noise: when extracted data from a code consists of classes and methods and inclusions and calling relations. Each element plotted on a grid, and each relation drawn as a line. Accurate view, but provides no insight into structures →


How to filter information?

Manipulating views (e.g. hiding methods inside class definitions)

Showing abstractions (e.g. identifing distinct layers)

Database construction III

Because of white noise → therefore manipulating such raw views to collapse information (ex. Hiding methods inside class definitions)


Views which have been extracted are manipulated to create fused views

e.g. static call view fused with dynamic call view to provide more accurate information about the part of system

Creating a fused view = creating a hypothesis about the architecture and a visualization of it

- views can be interpreted, further refined, or rejected

- no universal completion criteria for this process

View fusion


● Tool for static code review written in Java, support of other programming languages planed

● „Bridging the gap between the architects and developers“

● More http://www.hello2morrow.com/products/sonarj




→ Debt index: aggregated metrics to explain the effort to make system „clean“, „how difficult to remove all warnings and violations“

→ then lines of code etc.→ cyclicity, 30,98% of packages involved in cyclic dependencies.

By clicking on one, visualisation of cycle group will be shown→ below „how many spots to cut“ - to remove to package cycles→ other interesting metric: ACD (average component dependency)

shows how many files are going to be affected if I change this project

→ in warning box, general warnings e.g. 1 cycle group or code duplications (by clicking on it, system directly jumps to duplicated lines)


SonarJ extracts facts from a set of source code files and lets you define a set of layers and vertical slices through those layers.

View fusion: example of a fused view

Early results from the tool SonarJ.


View fusion phase gave us a lot of hypotheses– those inform of architectural elements (layers),

constraints, relationships

– Hypotheses need to be tested in order to check conformance with architect's intention

– Two major possibilities for maintaining conformance between code and architecture:

- conformance by construction

- conformance by analysis

Architecture analysis: finding violations

- by construction: automatically generation a substantial part of the system based on an architectural specification. Highly desirable, because tools guarantee conformance. Can be unfortunetely only applied when engineers can employ specific architecture-based development tools, languages. For systems requiring implementation from outside, this approach does not apply

- by analysis – technique to analyze system information to mark nonconforming elements, so they can be fixed: brought to conformance.


Previous figure with added relationships and constraints, which have been added by an architect to reflect design intent.

Architecture analysis II: example

In this example, the architect has indicated the relationships between the layers.

Using these relationships and constraints, a tool such as SonarJ is able to automatically detect and report violations of the layering in the software.

What does the figure tell us?

Data layer (row 2) can access, and hence depends on, the DSI layer.

Data layer may not access, and has no dependencies on, Domain, Service, or Controller (rows 1, 3, and 5).

The JUnit component in the “External” component is defined to be inaccessible.

No portion of the application should depend upon JUnit, because this should only be used by test code.


Black Arc: SonarJ has found a violation. The arc between the

Service layer and JUnit.

Architecture analysis III

This violation is found by SonarJ by searching through its database, applying the user-defined patterns, and finding violations of those patterns.

The highlighted arc indicates that this is an illegal dependency and architectural violation.


● Have a goal and a set of objectives or questions in mind before undertaking an architecture reconstruction project.

● Obtain some representation of the system before beginning the detailed reconstruction process.

● Identifying layers is a good place to start● In some cases: disregard the existing documentation● Tools can not do all the work! Involvement of people familiar

with the system required.

Guidelines


● Architecture reconstruction ensures that a system is built the way it was designed and that it evolves in a way that is consistent with developer's intentions.

● Reconstruction needed when:

- documentation lost or out-of-date

- checking conformance

- starting the reengineering process to a new desired architecture

Summary


● The software architecture reconstruction process comprises the following phases:

- raw view extraction

- database construction

- view fusion

- architecture analysis

Summary II


1. What inferences can you make about a system's module structures from examining a set of behavioral traces gathered dynamically?

Discussion

→dynamic analysis is performed by executing programs on a real or virtual processor

→ to be effective, sufficient test inputs needed; different techniques to ensure that a slice of set of possible behaviors has been observed e.g. code coverage (higher coverage, less bugs)

→ several steps: preparing input, running a test program, gathering parameters, analyzing the output

→ dynamic analysis is executed by passing a set of data to the input of the program : efficiency of test depends on quantity and quality of data

→ Metrics provided: - ressources consumed – time of program execution on whole or for

different modules, the number of external queries (e.g. to the database), the number of memory being used, other ressources

- cyclomatic complexity, degree of code coverage with tests, program metrics (on performance, reaction time, response)

- program errors (divisions by zero), null pointer dereferencing, memory leaks, race conditions (output depends on sequence or timing of other event)

- vulnerabilities in the program- other advantages: dynamic analysis detects dependencies that is not

possible for static, e.g. dynamic dependencies, polimorphism

Remember: even 100% coverage doesnt mean there are no errors at all! Dynamic analysis doesnt reveal logic errors and secondly, testing utility is not free of errors too.


2. Suppose you hypothesize that a system makes use of particular tactics to achieve a particular quality attribute. Fill in the columns of the table below to show how you would go about verifying your hypothesis.

Discussion

Tactics for.. Interviewing experts on the system

Analyzing structure of source code files

Static analysis of source code

Dynamic analysis of system's execution

Availability

Interoperability

Modifiability

Performance

Security

Testability

Usability

→ Interviewing: availability („what are the SLA? What time is the system meant to be

aviailable? What's the minimum availiable time? What's the maximum downtime? What's the counted % of availability?“)

Performance/Usability: user behavior analysis, surveyInteroperability (work with other systems): does the system adhere to

interface standards? Is there an easy way to convert into different standard?

→ Structure of code: availability/performance („Bad smells“ which or duplicated code, which lead to delays)

Security (security checks and detecting security vulnerabilities using tools) Interoperability: known standard used? Same format? Same codification of

data?

→ Dynamic analysis of system's execution: availability (response time, downtime/delays due to late binding, causing availibility problems, runtime measurements)

Performance: ressources consumed (how much time needed to execute a certain module?)

Testability: memory leaks

→ static and dynamic analysis: tools designed specifically for finding security flaws e.g. sql-injection, spoofing and other security vulnerabilities. Leading tools e.g. Fortify or Coverity.

Top security vulnerabilities e.g. unvalidated parameters, broken access control or sessions, insecure use of cryptography.

SoftwareReconstruction - Chapter 20 - Software Architecture in Practice

Documents