Evolutionary architectures and fitness functions: A pragmatic ......(ASP.NET WebAPI) HTTP/REST HTTP/REST WeatherApp Problems-Dependency on 3rd party service-Resilience measures not

Evolutionary architectures and fitness functions: A pragmatic approach

Software Architecture

#OReillySACon

Rene Weiss Berlin, 2019

AgendaEvolution!What are Fitness Functions?Examples! (#Code)How to start?

Rene Weissrw@embarc.de

xing.to/rweiss

@renebiancoSoftware Architect, Consultant, Trainer @ embarc

Focus areasq Software architecture (review, design, …)q iSAQB trainer for agile software architecture

(AGILA)q Agile methodologiesq (.NET technologies)

Founded 2013 in Hamburg

Since 2015 also in Vienna

IntroductionEvolutionary architectures

Fitness Functions

Nature has solutions for complex problems…

Evolutionary approaches….

… by using small, but steady adaptions

“Software architecture is about the important stuff, whatever that is.”

(Martin Fowler)

… what about architecture

Evolutionary architecture

Evolutionary architecture supports small, incremental changes spanning multiple dimensions of the software architecture.

q small changesq broad involvement with a lot of participantsq geared toward common goals

à Fitness Functions!

Fitness Functions!DefinitionCategorization (why categorize?)Tool overview

Fitness FunctionsBorrowed from evolutionary computing, a fitnessfunction is an objective function, used to summarize how close a given design solution is to achieving the set aims.

q On the search for improvements, the Fitness Function defines what is „better“.

q Equally „good“, but simpler approaches are superiour

Neal Ford, Rebecca Parsons, Patrick Kua

First Fitness Functions examplesProblem – Maintainability (Testability)� Is our test coverage sufficient?

Target audience:� Developers

Implementation idea:� Periodic review of test

coverage

Fitness Function(s):

§ Test coverage (Unit) > 0.9

§ Test coverage (Integration) > 0.6Timing:

� Each CI build� Each nightly integration test build

First Fitness Functions examplesProblem – Reliability (Availability, Resilience)� Is our service up and running if latency increases?

Fitness Function(s):

§ Integration test errors = 0% (set network latency for 3rd party connections = 5s)

Target audience:� Developers, Operations

Implementation idea:� Periodic tests of our

service with increased latency

Timing:� Each nightly integration test build

CategorizationExecution type§ Triggered § Continuous

Breadth of feedback§ Atomic § Holistic

Execution location§ CI/CD § Test environment§ Production

Quality attribute§ Maintainability§ Reliability§ Performance§ …

Monitoring type§ 0/1§ Value/Threshold§ Trend

Test applicability§ Temporary§ Permanent

Software quality attributes (-ilities)

Functional Suitability

Performance Compatibility Usability

Realiability Security Maintainability Portability

(ISO 25010)

Static code analysis (coupling, cyclomatic complexity, ..)

Functional tests + test coverage…

Autonomous runtime tests Immediate quality inspection

“Chaos Engineering”…

Consumer Driven ContractsExploratory testingPerformance testsQualitative analysis

…

LoggingMonitoring (transactions, queue

length, ressource usage …)...

Fitness Function Quadrantatomic

holistic

continuoustriggered

Fitness Function Quadrant - Toolingatomic

holistic

triggered continuous

Examples!A / B code testingMini-Chaos Engineering

< / >

Scientist (A / B Testing)Main Component Calculation Lib

Problems- Legacy (code not maintainable, „no“ test

cases)- Developers not available anymore- Errors are raised by various data constellations- Very hard to test – extremly large number of real world

test cases

Business-logic

Scientist (A / B Testing)

Solution idea

- Develop new library- Run experiments with new library in production- Continuous improvement through real world use cases

Fitness Function(s):

- Library V2 Result == Library V1 Result- Average: Lib. V2 Performance (processing time) -30% (vs. V1) - Worst Case (Single Result): Lib. V2 Performance (processing time) +10% (vs. V1)

Scientist (A / B Testing)

Main Component Calculation Lib. V1

Calculation Lib. V2

Scientist Experiment

Business-logic

Scientist Demo!

Scientist (A / B Testing)

Execution type: Continuous

Quality attribute: Funct. Suitability Performance

Monitoring type: 0/1Trend

Breadth of feedback: Atomic

Timing + Location: Production

Test applicability: Temporary

Fitness Function(s):

- Lib. V2 Result == Lib. V1 Result- Average: Lib. V2 Performance (processing time) -30% (vs. V1) - Worst Case (Single Result): Lib. V2 Performance (processing time) +10% (vs. V1)

Mini-Chaos Engineering

OpenWeather-Map

Client

Server(ASP.NET WebAPI)

HTTP/REST

HTTP/REST

WeatherApp

Problems- Dependency on 3rd party service- Resilience measures not testable

Mini-Chaos EngineeringSolution idea

- Circuit Breaker using Polly.NET- „Chaos“ (HTTP errors, latency, …) using Simmy- Tests running in integration test environment

Fitness Function(s):

- Integration-Test Error = 0% (network latency for 3rd party service = 5s)

Mini-Chaos Engineering

OpenWeather-Map

Client

Server(ASP.NET WebAPI)

HTTP/REST

HTTP/REST

WeatherApp

“CircuitBreaker” & fallback logic

Test for failure:

Chaos Demo!

Mini-Chaos Engineering

Execution type: Triggered(Continuous)

Quality attribute: Reliability

Monitoring type: 0/1

Breadth of feedback: Holistic

Timing + Location: Test env.(Production)

Test applicability: Permanent

Fitness Function(s):

- Integration-Test Error = 0% (network latency for 3rd party service = 5s)

How to start?

Initially

5. Automate testing and evaluation

1. Identify central quality goals (use ISO 25010)

2. Develop quality scenarios

3. Select top priority for evaluation / testing

4. Develop + formulate Fitness Functions

6. Visualize results

Continuouslye.g. every sprint

Add new, change or decommission unnecessary Fitness Functions(4., 5., 6.).

Step

Add quality goals à risk driven approach (1.)

Add scenarios(2., 3.)

Summaryq Evolutionary architectures: Small incremental architectural

changesq Direction of change guidance by Fitness Functionsq Fitness Functions can help in environments not using evolutionary

architecture approach

q Fitness Function categorization helps to generate ideas in solutionspace

ÞA lot of possibilities to test architecture!q Examples:

q A / B Testing on code level (libraries, …) incl. performance testing (Scientist.net)q Mini-Chaos engineering (Circuit-Breaker: Polly, Chaos-Testing Tool: Simmy)

Don’t automate (architecture) tests because you can…

… but because it makesmost sense in yourbusiness environment!

Rate today’s session

Session page on conference website O’Reilly Events App

Thank you very much!

Slide download: https://embarc.de/blogDemo source code: https://git.io/JegFX

rw@embarc.de

xing.to/rweiss

www.embarc.de

@renebiancoAny Questions?

Ressources

q Sample source codehttps://git.io/JegFX

q Pollyhttps://github.com/App-vNext/Polly

q Simmyhttps://github.com/Polly-Contrib/Simmy

q Scientist.nethttps://github.com/scientistproject/Scientist.net

q Scientist Case Study (GitHub)https://github.blog/2015-12-15-move-fast/

q Building Evolutionary Architectures. Support Constant Change N. Ford, R. Parsons, P. Kua, Patrick | O'Reilly 2017

q Software Architecture in PracticeChapter 4 (Understanding Quality Attributes, Quality Scenarios)Len Bass, Rick Kazman, Paul Clements | SEI Series in Software Engineering 2012

q Thoughtworks / Fitness Functionhttps://www.thoughtworks.com/search?q=fitness+functions&c=sitewide