Top Banner
Time to Build and Test Results 3x faster How We Did It Aurélien Pupier R&D Engineer, Studio Development Leader @apupier ©2014 Bonitasoft
26

Time to build and test results 3x faster - how we did it

May 25, 2015

Download

Technology

Aurelien Pupier

After one year of regular work on our build mechanism and Continuous Integration infrastructure, we are able to provide RCP products, along with their complete test suite, more than 3 times faster than before.

And guess what, we are using less physical resources.



The path to this result was quite long and sometimes tedious, but we finally managed it. I will share our experience and provide tips so you can speed up your own build. I will also mention which improvements are coming.



I will explain the methods used to identify bottlenecks and which technical solutions we applied (and the ones we dropped). This talk will go through the usage and possibilities brought by Tycho, Target Platforms, Hudson/Jenkins with some of its plugins and a Maven Repository Manager.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Time to build and test results 3x faster - how we did it

Time to Build and Test Results

3x faster – How We Did It

Aurélien Pupier R&D Engineer, Studio Development Leader

@apupier

©2014 Bonitasoft

Page 2: Time to build and test results 3x faster - how we did it

Agenda

• Initial context

• Actual status

• Methodology and Technology

• What’s next?

©2013 Bonitasoft 2

Page 3: Time to build and test results 3x faster - how we did it

Initial context

Once upon a time…

©2013 Bonitasoft 3

Page 4: Time to build and test results 3x faster - how we did it

What?

©2013 Bonitasoft 4

Page 5: Time to build and test results 3x faster - how we did it

What it means?

©2013 Bonitasoft 5

Page 6: Time to build and test results 3x faster - how we did it

Global Infrastructure

©2013 Bonitasoft 6

φ Linux

SSD

VM

- Hudson slave

- RCP builds

φ Mac

- Hudson slave

φ Linux

VM

- Hudson Master

- Other Bonita

components

Page 7: Time to build and test results 3x faster - how we did it

Execution time

©2013 Bonitasoft 7

Page 8: Time to build and test results 3x faster - how we did it

Actual status

What do we gain?

©2013 Bonitasoft 8

Page 9: Time to build and test results 3x faster - how we did it

3x faster!

©2013 Bonitasoft 9

Page 10: Time to build and test results 3x faster - how we did it

Simplicity!

©2013 Bonitasoft 10

φ Linux

SSD

- Other Bonita components

- RCP builds

Page 11: Time to build and test results 3x faster - how we did it

Methodology and Technology

Through the path

©2013 Bonitasoft 11

Page 12: Time to build and test results 3x faster - how we did it

Workforce

• Along the way

• 1 year

• Dedicated part-time “Build engineer” at the

end

©2013 Bonitasoft 12

Page 13: Time to build and test results 3x faster - how we did it

Tycho

• PDE build Tycho

• Easier Modularization

• Help from non-Eclipse RCP developer

– 2 20+

• Easier for Bonita Community

©2013 Bonitasoft 13

Page 14: Time to build and test results 3x faster - how we did it

Modularization

• Easier to split

• On-demand/nightly build:

– Translations

– Patched plugins

– Generated code

– Installers

©2013 Bonitasoft 14

Page 15: Time to build and test results 3x faster - how we did it

Mirrored Target Platform

• Avoid network latency

• Remove external dependencies

– LTS

– Stay the master

©2013 Bonitasoft 15

Page 16: Time to build and test results 3x faster - how we did it

Remove virtualization

• I/O access

• Virtualization killed SSD performance

• Tried

– mounted partition

– Logical Unit

©2013 Bonitasoft 16

Page 17: Time to build and test results 3x faster - how we did it

Backup

©2013 Bonitasoft 17

• Environment and CI config stored on

– Less down time

– Easier to modify environment

Page 18: Time to build and test results 3x faster - how we did it

Jenkins Scriptler plugin

• Allow “complex” build flow

• Easier to test a subpart

• Rejected:

– build-pipeline-plugin

– join-plugin

– Downstream/Upstream

©2013 Bonitasoft 18

Page 19: Time to build and test results 3x faster - how we did it

Don’t sleep while testing!

• bot.sleep() bot.waitUntil()

• Gain:

– Stability

– Speed

©2013 Bonitasoft 19

Page 20: Time to build and test results 3x faster - how we did it

Genisoimage

• Generate .dmg

©2013 Bonitasoft 20

φ Mac

- Hudson slave

Page 21: Time to build and test results 3x faster - how we did it

The key of success

©2013 Bonitasoft 21

Page 22: Time to build and test results 3x faster - how we did it

The key of success

©2013 Bonitasoft 22

• Fail Fast

– Easy roundtrip

– Subpart testing

Page 23: Time to build and test results 3x faster - how we did it

What’s next?

Harder, Better, Faster

©2013 Bonitasoft 23

Page 24: Time to build and test results 3x faster - how we did it

Others

• Tycho

– TP resolution

– Young project

• Test split

– Slow/fast

– Enough confidence on a smaller set?

• Translations

– BND use

– Avoid TP resolution

• Workflow Integration in Hudson/Jenkins

©2013 Bonitasoft 24

Page 25: Time to build and test results 3x faster - how we did it

©2013 Bonitasoft 25

Page 26: Time to build and test results 3x faster - how we did it

Evaluate This Session

Sign-in: www.eclipsecon.org

Select session from schedule

Evaluate:

1

2

3