COSMIC Annual Report 2014

ANNUAL REPORT 2014 Method Maturity & Organizational Formalization

May 2015

Executive summary For COSMIC, 2014 is the year in which we upgraded the method to version 4.0. The same principles have now become more accessible to novice users and non-native English speakers

We also worked hard to make the organization more professional. New legislation in Canada speeded up the organizational part, because we had to rewrite our Constitution to fulfill the obligations posed by the new Not For Profit act in Canada. All key positions in the COSMIC organization are now subject to a 3-year review/re-election period to ensure that people holding such a position remain active and committed to the organization and its goals.

In 2014 COSMIC dedicated a lot of time and energy in the relation with national Software Metrics Associations. Now key officials of a number of national SMA’s also hold key positions in the COSMIC organization. Among them the United States, Brazil, Mexico, Germany, Italy, Poland and the Netherlands. This has also resulted in two combined projects: - The development of a Case Study, together with Nesma - A common glossary on NFR, together with IFPUG

We also worked hard in realizing an on-line certification exam for the entry-level certification exam. In this way more people can prove their knowledge of the fundamentals of the COSMIC method.

Also we welcomed two additional countries to the IAC: Australia and South Africa. Two important industrialized nations now also have a local representation from COSMIC.

A brief overview of 2014 The COSMIC method is designed to be future-proof. In 2014 we again witnessed this, with more countries joining the COSMIC organization, more people getting certified and a lot of activities on various continents to translate COSMIC documentation to a local language, to help facilitate the adoption of the method for whom English is not their native language.

Future-proof is not the same as: no developments. Although the method has been unchanged since the COSMIC method became an International Standard., the growing use of the method revealed that some parts of the Measurement Manual needed more careful wording. Novice users and translators found out that some parts could be improved on the ease of understanding the method. With the release of version 4.0 of the Measurement Manual, our core document was thoroughly improved and easier to understand. With the Chinese translation the method became also available to a large potential userbase in their own language, and many other translations will follow.

We also introduced a new document ‘Introduction to the COSMIC method’ which is aimed at decision makers and other people that need to have a quick understanding of the general principles and advantages of the method.

The Business Application Guideline has been upgraded to version 4.0 and the work on upgrading the other existing guidelines has started. The MPC started work on developing new Guidelines as well on early and rapid sizing, convertibility, and on how to account for Non-Functional Requirements.

A brief overview of 2014 With this growing use and adoption, it was also time to transform the COSMIC organization to become more professional. When COSMIC started in 1998, no formal rules were necessary. In 2011 COSMIC was already big enough to have a limited set of rules and we formalized the first version of our Constitution. The new Not For Profit Act in Canada, our legal homebase, was a good moment to bring the organization a step further with the formalization of a review/re-election cycle for most of the key positions in the organization.

We also adopted a new logo and corporate style, to bring our “corporate” image in line with today’s demands to make sure that the way we present ourselves to the world has the look and feel to remain future-proof.

In this Annual Report we show some of the figures underlying the continuing use and adoption of the COSMIC method for functional size measurement. The major part of this report has been reserved for where it all happens: all over the world!

Alain Abran, Chairman Frank Vogelezang, President

Downloads of the Manual One of the key indicators of the use of the COSMIC method is the number of downloads of the core document of the COSMIC method, the Measurement Manual:

Registered charity in Canada The legal status of the COSMIC organization is that we are a registered charity under Canadian law. In 2013 the new Not For Profit Act came into effect and in order to keep our legal status we had to renew our Constitution to be in line with the new Act.

The new Act had a lot of procedures for not for profit organizations as default, so the Constitution did not have to describe a lot of things, unless we had different requirements. One of the things that was mandatory under the new Act, was to describe how people would start and end an official position within the COSMIC organization. Therefore we introduced “terms of office” for most permanent key positions in the organization, like the IAC and the MPC. This allows the organization to keep these positions future-proof every three years, without introducing a full bureacratic institution.

One of the other things we now made official is that COSMIC is a voluntary organization that maintains an Open Source method for functional size measurement. We believe that this is, next to the firm scientific foundations of the method, one of the most important aspects of being and remaining fit for the future.

Entry-level certification In 2014 we had 75 new people who gained their Entry-level certification. Most of them are from Poland:

Up to the end of 2014, 485 people had obtained their Entry-level certification.

A new style and website One of the things that is vitally important in spreading the method is to present ourselves the way practitioners want to find us. This meant that our website would need to be able to present the information not only on a regular website, but also on Mobile devices like tablets and phones. Also our URL needed to be in line with our organizational status, so a .org address instead of .com. We also decided that we would need a strong logo.

The new logo and “corporate” style are already in use. This presentation is a result of that. The new URL will be www.cosmic-sizing.org and will be available in the first half of 2015.

The new COSMIC logo The icon is a stylized version of a drafting compass, representing the key part of the COSMIC organisation: measurement. The drafting compass measures the distance between the 0 and 1 in the COSMIC acronym, symbolizing the software domain in which the COSMIC method operates. The fact that the stylized drafting compass is symmetrical stands for the balanced nature of the method, that is founded on solid software engineering principles.

The exact nature of the COSMIC method is further accentuated by the color combination of blue and dark fuchsia. The two legs of the drafting compass are positioned in an overlay to create depth. The use of a gradient in the colors adds to this effect. It also creates the effect that the legs seem to be walking forward. This effect is enhanced by the arc in the icon, that stands for the upward and forward trend in the development of the COSMIC method and its use. The round characteristics of the font that is used in the wordmark, combined with the sharp and solid lines used in the icon is distinctive and creates a solid ground for the rest of the corporate identity.

The logo should always be used as the combination of the icon and the wordmark. The icon should only be used in social media where there is a distinction between an icon and a full logo.

COSMIC developments throughout the world

Country reports

Australia

Agile productivity using COSMIC At Aon, a global risk broker, COSMIC is used to drive the agile development process.

All work is driven through the lense of COSMIC, which gives unparalleled insight into the productivity and therefore, the predictability.

Belgium

COSMIC Training Part of Software Quality and Measurement – Université de Namur

One commercial training by CETIC

One workshop at the IWSM-Mensura conference in Rotterdam

Industry adoption no global data but a majority of large companies

2 extra companies were introduced to COSMIC in 2014

Belgium

Promotion CETIC is preparing a survey on the use of estimation methods. Its objectives are:

to get a first status on who is using what in Belgium

to adapt the COSMIC offer to the requirements of Belgian companies

An industrial discussion group on COSMIC will be organized in 2015

CETIC is planning to issue at least one publication in 2015, possibly in an international conference and reporting about its experience from industrial field

CETIC is thinking on how to integrate the use of COSMIC in a more general quality-oriented process including the ISO29110 standard, quality of code evaluation and goal-oriented requirements engineering

Brazil

Training Two public training courses

One webinar

Two measurement companies training their staff and customers

Brazilian/Portuguese translation of the Measurement Manual started

Certification Participation with one candidate in the online certification exam

Two new certifications

Brazil

Publicity Meetings and discussions with FSM power users (big companies)

Two measurement companies teaching COSMIC in Brazil

Active participation in the LinkedIn COSMIC Forum

Challenges Both private companies and the government have already invested a lot of resources

on making IFPUG FP their main metric and don’t feel the urgency to change that

C-level executives are not very sensitive to COSMIC advantages and prefer to maintain the status quo. Change may happen but it may also be slow

SNAP has recently drawn the attention of the Brazilian market

Canada

Courses that include COSMIC Project Management (how to use COSMIC for estimation) at ÉTS and UQAM

Software Requirements (how to use COSMIC for evaluating the quality of the requirements) at ÉTS

Software measurement course at UQAM

2 graduate students in using the COSMIC method for their master's project.

Promotion of the COSMIC method Translation of the Measurement Manual v4.0 to French

Creation and implementation of the certification exam

Canada

Publications Dumas-Monette, Trudel, ‘Requirements Engineering Quality Revealed through

Functional Size Measurement: An Empirical Study in an Agile Context’, IWSM 2014

Abran, Desharnais, Zarour, Demirors, ‘Productivity Based Software Estimation Model: An Economics Perspective and an Empirical Study’, 9th International Conference on Software Engineering Advances - ICSEA 2014, Nice (France),

Soubra, Abran, ‘Verifying the Accuracy of Automation Tools for the Measurement of Software with COSMIC – ISO 19761 including an AUTOSAR-based Example and a Case Study’, IWSM-MENSURA 2014

Valdès, Abran, ‘COSMIC Approximate Sizing Using a Fuzzy Logic Approach: A Quantitative Case Study with Industry Data’, IWSM-MENSURA 2014

PhD Thesis Monica Villavicencio: Development of a framework for the education of software

measurement in software engineering undergraduate programs

Ecuador

Presentations about COSMIC Carlos Monsalve´s presentation as keynote speaker @ a national conference organized

by a University in Esmeraldas-Ecuador. The conference included topics on Programming and Software Development [LINK]

Carlos Monsalve´s presentation as keynote speaker @ CIISTEDG-FIE 2014 conference. A national conference organized by a University in Riobamba-Ecuador [LINK]

Carlos Monsalve´s presentation as keynote speaker @ COINISC 2014 conference. An international conference organized by IEEE and a Peruvian University. [LINK]

Mónica Villavicencio´s presentation as keynote speaker @ Software engineering conferences at UPS (Guayaquil – Ecuador) [LINK]

Institutions teaching COSMIC Escuela Superior Politécnica del Ecuador ESPOL

Universidad Politécnica Salesiana (UPS – Guayaquil)

Germany

Book including COSMIC Dumke, R.; Schmietendorf, A.; Seufert, M.; Wille, C.: Handbuch der

Softwareumfangsmessung und Aufwandschätzung. Logos-Verlag, Berlin, 2014, (570 S.), ISBN 978-3-8325-3784-5 (including COSMIC FP 4.0)

Publicity CECMG workshop in March from Andreas Schmietendorf

BSOA/BCloud workshop in Frankfurt in November from Andreas Schmietendorf

DASMA/GI Metrikon conference in November

Reiner Dumke is editor of the Software Measurement News where one page for COSMIC news can be reserved

Italy

Training & certification 1 COSMIC training (v3)

1 COSMIC exam (v3)

6 foundation-level certificates (4 v3 and 2 v4)

Pilot online certification exam

Known use of the COSMIC method CSI-Piemonte (baseline, MIS with IFPUG; Batches, GIS, DWH with COSMIC)

Central PA inhouse Software House (concluding 2013 pilot with COSMIC; positive results on Batches and BPM-based product)

Central PA Welfare Inst. INPS (Guideline and experimentation for ERP/SAP projects; COSMIC standard for developments + local extension for Parametrization; audit supported on COSMIC measures ERP and Mobile)

Italy

Publicity Presentations of COSMIC news updates at 3 GUFPI-ISMA meetings that were also

open to non-members

COSMIC news through GUFPI-ISMA news mailing in about 10 issues (mailings are also open to non-members)

Initiated MM v4 translation to Italian

Delivered 2 free-of-charge presentations on the COSMIC method

Challenges Major historic support of IFPUG (including SNAP for NFR’s)

SIFPA promoted as a better FSM method at different meetings

Lack of benchmark data for COSMIC

Historical presence of IFPUG practitioners and productivity baselines

Mexico

Publication about COSMIC Francisco Valdés, Alain Abran, ‘COSMIC Approximate Sizing Using a Fuzzy Logic

Approach: A Quantitative Case Study with Industry Data’, IWSM-MENSURA 2014, Rotterdam (Netherlands), Oct. 6-8, 2014, IEEE Press, pp. 282-292 DOI 10.1109/IWSM.Mensura.2014.44

Training & certification 1 conference in Academic environment

3 webinars including the COSMIC topic.

3 COSMIC Courses via COSMIC SIG in Mexico

2 COSMIC certification exams in 2014 (1 physical, 1 by Internet) 6 COSMIC v.4. entry-level certificates (CCL) in 2014

5 COSMIC v.3.0.1 entry-level certificates (CCL) in 2014

Mexico Regulation including COSMIC Release of the Mexican Information Technology and Communications and Information

Security Process Framework for the Government; known as (MAAGTICSI), that includes the NMX-I-119 in the Project Management Process, equivalent to ISO 19761

Beginning of the Initiative "Big Projects suppliers for government"

Promotion Special Interest Group on COSMIC / Mexico , www.sigcosmicmx.wordpress.com

Increase the COSMIC certificate holders

Plan to publish a COSMIC articles/Information via SPINGERE and Special Interest Group on COSMIC / Mexico Blogs

Organize a Software Measurement and Estimation Conference in Mexico

Organize a COSMIC courses

Participating in the project “Blueprint for the improvement of the Mexican Software Industry”

Realize a COSMIC Method Implementation in a governmental entities trough the authorized vendor in Mexico

the Netherlands

IWSM-Mensura 2014 Organization of the IWSM-Mensura conference 2014 in Rotterdam, many COSMIC

related activities like the annual meeting, workshops, presentations and certification exams. COSMIC paper from the Netherlands:

Measure the functional size of a mobile app using the COSMIC FSMM van Heeringen and van Gorp

Training Nesma offered COSMIC to use its extensive hotel case as a case study for the COSMIC

method as well.

COSMIC principles covered in 3 guest lectures at Dutch universities

the Netherlands

Opportunities The Dutch government issued a report in which estimation and control based on

functional size are an important recommendation to avoid cost overruns in government projects;

The Dutch IT industry is going agile. Multiple papers show that COSMIC is often a better fit to size the documentation in agile teams than other FSMM.

The industry needs a method to size NFR. The COSMIC initiative could provide a more useful method than the SNAP approach.

Challenges Most organizations that use functional sizing in the Netherlands use the Nesma FPA

method. Since they are mainly administrative organizations there is not much urgency to progress to COSMIC.

Benchmarking organizations like Gartner push back large organizations that used COSMIC into FPA.

Poland

Publications Beata Czarnacka-Chrobot, Analysis of the Per-unit Work Effort and Per-Unit Work Cost

of the Web Information Portal Enhancement – a Case Study from Poland, IWSM-Mensura 2014, October 6-8, 2014, Rotterdam, DOI: 10.1109/IWSM.Mensura.2014.13.

Beata Czarnacka-Chrobot, Analysis of the Workflow System Ehancement Project Speed and Duration – a Case Study, “Software Engineering”, vol. 4, no. 1, DOI: 10.5923/j.se.20140401.02

Courses that include COSMIC IT Project Management, Warsaw School of Economics (postgraduate)

Estimating Software Projects Costs, Warsaw School of Economics (postgraduate)

Conference organization Organization of the IWSM Mensura 2015 conference in Cracow

Spain

Training Part of the Software Engineering subject at University of Alcala

Actions to promote the COSMIC method There is not much knowledge about the COSMIC method in the academic environment nor in the Industry. We should try to focus onto showing the Methodology and its benefits to the clients we have and to try to teach it in some Spanish University as part of the “Project Management” subject, for example.

Spain COSMIC-related publications Nelly Condori-Fernández, Sergio España, Klaas Sikkel, Maya Daneva, Arturo González:

Analyzing the Effect of the Collaborative Interactions on Performance of Requirements Validation. REFSQ 2014: 216-231.

Sebastian Bauersfeld, Tanja E. J. Vos, Nelly Condori-Fernández, Alessandra Bagnato, Etienne Brosse: Evaluating the TESTAR tool in an industrial case study. ESEM 2014.

Alvarez Puertas, Luis; Álvarez Cabal, Jose Valeriano; Villanueva Balsera, Joaquín; Concepción Suárez, Ramiro: Control Methods in Agile Project Management Office. 18th International Congress on Project Management and Engineering, Alcañiz, Spain, 16-18th July 2014.

N. Condori-Fernández, Peter M. Kruse (Berner & Mattner), TEJ Vos, E. Brosse, A. Bagnato. Combinatorial Testing in an Industrial Environment–Analyzing the Applicability of a Tool. 9th Internation Conference on the Quality of Information and Communication Technology (QUATIC) 2014.

Granda, M.F., Condori-Fernandez, N. ; Vos, T.E.J. ; Pastor, O. Towards the automated generation of abstract test cases from requirements models. IEEE RET Workshop. Sweden.

Yeshica Isela Ormeño, José Ignacio Panach, Nelly Condori-Fernández, Oscar Pastor: Capture usability requirements through guidelines in Model Driven environments. Special Issue of International Journal of Information System Modelling and Design (IJISMD)

Switzerland

COSMIC events ICTscope.ch is the Software Measurements & Metrics expert group for SwissICT, the Swiss ICT association, S-I, the Swiss association for computer scientists, and SPM, the Swiss association for Project Management (IPMA).

In 2014 six meeting were held, 2 on COSMIC:

Automatic COSMIC measurements from UML sequence diagrams

COSMIC for Scope Management and Testing

COSMIC projects Excel-based application for measuring UML Sequence Diagrams with COSMIC

On hold: Swiss Railways developed a method for automatic COSMIC measurements based on UML Use Case analysis for the full project life cycle

Switzerland

Conference contributions on COSMIC COSMIC versus IFPUG for measuring Mobile Apps

(GUFPI-ISMA)

Using COSMIC in the Buglione-Trudel Matrix to predict project cost with regards to functional and non-functional requirements (XP2014 Rome)

Requirements Elicitation measurements based on COSMIC (RE’14 Karlskrona)

Managing Projects based on COSMIC measurements (SharePoint Conference Düsseldorf)

Defect Density Measurement with COSMIC (Rotterdam)

Measuring Defects with COSMIC at 5th International Lean Six Sigma conference (Edinburgh)

Measuring Tests with COSMIC at IT Confidence conference in Tokyo

Turkey

Industry adoption A leading GSM company in Turkey has initiated a project to measure its software functionality with COSMIC. As they are a leading IT company and they are also using outsource services, we are observing an increase in COSMIC training and certification requests from smaller IT organizations.

Courses that include COSMIC Software Management (SM511) Middle East Technical University, Ankara

Software Project Management (IS502) Middle East Technical University, Ankara

Turkey

Publications based on Turkish COSMIC data IWSM-Mensura 2014

Banu Aysolmaz and Onur Demirörs. Automated Functional Size Estimation using Business Process Models with UPROM

Mert Ertugrul, Gökçen Yilmaz, Murat Salmanoglu and Onur Demirörs. Highlighting Error Categories in FSM Training

Muhammet Ali Sağ and Ayça Tarhan. Measuring COSMIC software size from functional execution traces of Java

UYSM 2014: (Turkish National Software Engineering Conference)

Önder Eren, Onur Demirörs, Barış Özkan. Yazılım Ürün Hatları için Otomatik İşlevsel Büyüklük Ölçümü Yaklaşımı

Murat Salmanoglu, Onur Demirors. COSMIC Fonksiyonel Büyüklük Ölçüm Yöntemi için Bir Hata Önleme Modeli

Erdir Ungan, Numan Çizmeli, Onur Demirörs. Scrum Uygulamalarında İşlevsel Büyüklük Tabanlı İşgücü Kestirimi ile Hikaye Puanı Tabanlı İşgücü Kestiriminin Karşılaştırılması

United Kingdom

Raising awareness Given the continuing low general level of interest in software metrics in the UK, marketing has aimed to raise awareness at senior levels which we hope will be effective in 2015.

A study group was formed to analyse the on-going failures of major public sector IT projects to deliver on time and budget, if at all. The group proposes to establish a new public sector body whose responsibilities would include the collection of performance data on IT activities which can be used to drive performance improvement.

A paper on ‘Bridging the gap between Chaotic and Controlled software project estimating’ will be published by the Society for Cost Analysis and Forecasting

A talk will be given to the British Computer Society Requirements Engineering Specialist Group on ‘How can Requirements Engineers help project estimators?’

Efforts continue to promote the COSMIC method within Central Government.

We understand that the excellent Renault experience of using the COSMIC method is spreading within the UK automotive industry.

United Kingdom

Special Interest Group A UK COSMIC Special Interest Group has been formed, with two well-attended meetings. Topics discussed include:

Experience of using COSMIC to control Agile projects in SITA (Société Internationale de Télécommunications Aèronautique)

Early work on defining what is meant by Non-Functional Requirements (NFR) and how to account for them.

The latter work continued with the development of a COSMIC ‘Guideline on how to account for NFR in software project performance measurement, benchmarking and estimating’. This is now being reviewed by the Measurement Practices Committee.

United Kingdom

Cooperation with IFPUG The Glossary of NFR terms from the Guideline has been shared with IFPUG with the

intention that it be published jointly. This first COSMIC/IFPUG collaboration is proceeding well.

Charles Symons gave a talk at the IFPUG ISMA9 conference in Madrid on ‘Measures and measurements to get service from your software suppliers’.

Presentations At the Rotterdam IWSM/Mensura 2014 conference:

Symons and Chris Woodward contributed to the COSMIC Masterclass on v4.0 of the method and on NFR respectively.

Symons presented on ‘Lies, damned Lies and Software Metrics (best presentation)

At the UK COSMIC Special Interest Group:

Experience of using COSMIC to control Agile projects in SITA (Société Internationale de Télécommunications Aèronautique)

United States

Reaching out Reached out to the COSMIC ‘community’ in the US to advance the idea of forming a US based user group to meet periodically and to discuss opportunities and challenges with using COSMIC. No measurable response to the idea in 2014.

www.cosmic-sizing.org