Www.modeliosoft.com Softeam’s testing process: a story Cyril Ballagny, SOFTEAM | ModelioSoft The 13th CREST Open Workshop London, England 13th of May 2011.

www.modeliosoft.com

Softeam’s testing process: a storyCyril Ballagny, SOFTEAM | ModelioSoft

The 13th CREST Open WorkshopLondon, England13th of May 2011

2

Introduction to Softeam

A French SME (about 230 people) More than 20 years of expertise in OO development and

Modeling Consulting, Training, Engineering activities Software Publishing: Modelio and Objecteering Strong R&D efforts since its creation

– Innovator since 1989 (Modeling, MDA, UML)– Heavily involved in R&D cooperative projects (FITTEST, MADES,

ENOSYS…)– Participates in the OMG standardization efforts since 1996 (UML

profiles, SOAML, SysML…)

www.modeliosoft.com

3

Introduction to Modelio

A CASE tool For UML modeling

For MDA application

For Code generation (Java,

C#, C++…)

For documentation

generation (HTML, word…)

For team work and project

sharing

And much more (BPMN,

SysML, SOAML…)!

www.modeliosoft.com

4

Modelio development process

Agile development with bootstrapping and continuous integration 20 people involved 1 million LOC (C++/Java) A new release every 6 months

www.modeliosoft.com

5

Modelio testing process (1)

Code quality management with Sonar: about 500 coding

rules

Continuous integration with Hudson: report broken build

(inconsistent API, missing source in the repository…)

Bug tracking with Mantis: everyone from the team can

report bugs

Specification of test plans with Test Link (natural language):

about 2000 procedures

Bootstrapping: Modelio is validated by developers

www.modeliosoft.com

6

Modelio testing process (2)

A test case is either A Mantis bug (bug 2123, bug 9478…)

A menu item (create, delete, mask…)

A diagram artifact (state, region, activity…)

Most test cases are UI-based and derived from general

scenarios (e.g. apply command “xxx” to element in

{a,b,c…} )

Test case execution is externalized to Eastern Europe

www.modeliosoft.com

7

Our issues

Specification often changes Writing operational test specification is too time consuming

compared with natural language specification

Test code must be tested too

Mix of languages (C++/Java/Jython) No tools for testing HMI combining Java and C++

www.modeliosoft.com

8

Our expectations (1)

Reduce over-testing Detect overlapping sections of test cases

Recombine sections to constitute smaller test cases with greater

coverage

Apply regression tests only on parts of the system that have

changed

Reduce under-testing Detect system parts that are not covered by the test cases

Detect sequences of interactions allowed by the GUI that have not

been tested

www.modeliosoft.com

9

Our expectations (2)

Reduce Time-To-Market Generate test cases instead of specifying them

Prioritize: Identify the most used features

Identify the most frequent bugs

www.modeliosoft.com

10

To sum up

We would like a scalable and pragmatic approach

whose ROI has been demonstrated

www.modeliosoft.com

Thank you for your attention!

Andrey SadovykhSOFTEAM | [email protected]

SOFTEAM R&D Web Site: http://rd.softeam.com

ModelioSoft Web Site: http://www.modeliosoft.com

www.modeliosoft.com 11

M o d e l i n g s o l u t i o n s.