ppbench A Visualizing Network Benchmark for Microservices Nane Kratzke and Peter-Christian Quint 1 Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems
ppbench A Visualizing Network Benchmark for Microservices
Nane Kratzke and Peter-Christian Quint
1 Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems
The next 15 to 20 minutes are about ...
• The cloud-native application stack and microservices, containers and container clusters (Docker Swarm, Mesos, Kubernetes, etc.)
• Possible technical and performance impacts of above mentioned approaches
• A benchmarking tool for figuring out performance impacts in the upfront of a microservice design.
• Some evaluation results of ppbench. • Some performance insights you might not know about Ruby, Java,
Go, Dart, Docker (container) and overlay networks like Weave!
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 2
Microservices
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 3
„The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechnisms, often an HTTP resource API.“
Martin Fowler
The Cloud-Native Stack (ClouNS)
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 4
(1) Different Programming Languages
(2) Overlay Networks
(3) Container
(4) VM Types
However, there is missing a tool ....
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 5
to figure out possible impacts of z different programming languages
z overlay networks
z additional container layer
z different VM types
z ...
in the upfront of a microservice design.
Reference Experiment
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 6
Experiment to measure reference network performance of REST-like HTTP-based communication protocols.
Ping and pong services are implemented in different TIOBE TOP 50 programming languages.
And we get visualizations ...
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 7
Data Transfer Rates(bigger is better)
Message Size (kB)
Tran
sfer
Rat
e (M
B/se
c)
0 kB 50 kB 150 kB 250 kB 350 kB 450 kB
0 M
B/se
c16
MB/
sec
32 M
B/se
c48
MB/
sec
64 M
B/se
c80
MB/
sec
● bare on m3.2xlarge For identifying - problematic areas - skewness of distributions - across a wide range of
message sizes
In that case. Dart‘s http standard libraries seem to have problems around message sizes bigger than three times TCP standard receive window (dotted lines).
Skewness is shifted to bottom (not preferable).
Skewness is shifted to top (preferable).
Impact of Containers
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 8
Experiment to measure container impact on network performance of REST-like HTTP-based communication protocols.
Impact of Software Defined Networks
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 9
Experiment to measure additional SDVN impact on network performance of REST-like HTTP-based communication protocols.
weave
Language Impact
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 10
Data Transfer Rates(bigger is better)
Message Size (kB)
Tran
sfer
Rat
e (M
B/se
c)
0 kB 50 kB 150 kB 250 kB 350 kB 450 kB
0 M
B/se
c22
MB/
sec
44 M
B/se
c66
MB/
sec
88 M
B/se
c
●
●
●
●
Go (Exp. P1) on m3.2xlargeJava (Exp. P2) on m3.2xlargeRuby (Exp. P3) on m3.2xlargeDart (Exp. P4) on m3.2xlarge
Requests per second in relative comparison(bigger is better)
Message Size (kB)
Rat
io (%
)0kB 50kB 150kB 250kB 350kB 450kB
0%20
%40
%60
%80
%11
0%14
0%17
0%20
0% ●
●
●
●
Reference: Go (Exp. P1) on m3.2xlargeJava (Exp. P2) on m3.2xlargeRuby (Exp. P3) on m3.2xlargeDart (Exp. P4) on m3.2xlarge
(1) Different Programming Languages
(1) Language impact should not be undererstimated
(2) Do not overestimate Go.
Overlay Network Impact
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 11
Data transfer in relative comparison(bigger is better)
Message Size (kB)
Rat
io (%
)
0kB 50kB 150kB 250kB 350kB 450kB
20%
50%
80%
120%
●
●
●
Reference (Exp. S1, S3) on m3.largeGo, SDN deployed (Exp. S2 compared with S1) on m3.largeRuby, SDN deployed (Exp. S4 compared with S3) on m3.large
(2) Overlay Networks
(1) OK, SDN decreases performance. Not astonishing.
(2) But, SDN can sometimes improve performance!
weave
Container Impact is likely to be hardly measurable in real world scenarios
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 12
Round−trip latency in relative comparison(smaller is better)
Message Size (kB)
Rat
io (%
)
0kB 50kB 150kB 250kB 350kB 450kB
90%
110%
130%
●
●
●
Reference on m3.xlarge (Exp. C1/C3)Dockerized Go (Exp. C2 compared with C1) on m3.xlargeDockerized Dart (Exp. C4 compared with C3) on m3.xlarge
(3) Container
(1) Container impact might be language dependent.
(2) Container impact is most of the times below 10%.
Overlay Network and VM Type Language impact is more severe than SDN
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 13
Data transfer in relative comparison(bigger is better)
Message Size (kB)
Rat
io (%
)
0kB 50kB 150kB 250kB 350kB 450kB
40%
60%
80%
100%
120%
●
●
●
●
Reference: (Exp. V1, V3, V5)SDN loss on m3.large (2−core, V2 compared with V1)SDN loss on m3.xlarge (4−core, V4 compared with V3)SDN loss on m3.2xlarge (8−core, V6 compared with V5)
(4) VM Types
Decreasing loss with increase of cores due to a reduction of CPU contention effects.
(1) SDN deployed on high core machines decreases performance down to 80% - 90%.
(2) That is far less than a potential language impact (40% - 190%).
Conclusion
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 14
• ppbench is a benchmarking tool designed to be used in the upfront of cloud-native application design processes.
• ppbench is provided via GitHub: https://github.com/nkratzke/pingpong
• You can run arbitrary experiments with ppbench to inspect possible performance impacts in the upfront of a service design/implementation.
• We presented some evaluation results of ppbench • SDN should always be used with high core machine types • Programming Language might be more performance
relevant than a SDN
Acknowledgement
• Stonehenge: Wikipedia, http://pt.wikipedia.org/wiki/Stonehenge • Soldier: Pixabay (CC0 Public Domain) • Microservices: Robert Morschel, http://www.soa-probe.com/2015/03/microservices-
summary.html
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 15
This study was funded by German Federal Ministry of Education and Research (Project Cloud TRANSIT, 03FH021PX4). The author thanks Lübeck University (Institute of Telematics) and fat IT solution GmbH (Kiel) for their support of Cloud TRANSIT.
Picture Reference
About
Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 16
Nane Kratzke
CoSA: http://cosa.fh-luebeck.de/en/contact/people/n-kratzke
Blog: http://www.nkode.io
Twitter: @NaneKratzke
GooglePlus: +NaneKratzke
LinkedIn: https://de.linkedin.com/in/nanekratzke
GitHub: https://github.com/nkratzke
ResearchGate: https://www.researchgate.net/profile/Nane_Kratzke
SlideShare: http://de.slideshare.net/i21aneka