Revealing Semantic Quality Characteristics of Free and Open Source Software Stergios Draganidis [email protected] Kerstin V. Siakas [email protected].

Revealing Semantic Quality Characteristics of Free and Open Source Software

Stergios [email protected]

Kerstin V. [email protected]

Department of Informatics Technological Educational Institute of Thessaloniki, Greece

Outline

This paper describes the initial stage of a research project at the department of Informatics of Alexander Technological Education Institute of Thessaloniki in Greece1

The research project investigates the quality of FLOSS2 considering underlying human factors that affect the six high level quality characteristics according to ISO-9126

1 This work is funded by the Greek Ministry of Education (25%) and European Union (75%) under the EPEAEK II program “Archimedes ΙΙ“

2 Free/Libre Open Source Software

Objectives

to introduce a scientific, rational and structured instrument for FLOSS product assessment

to correlate the quality characteristics described in ISO-9126 with measurable aspects and underlying factors of FLOSS collaborative development

to use the results from the initial stage for improvement of the survey instrument to be used on a larger scale

State-of-the-art

During the last decades a large number of FLOSS products have been developed

The development of software through collaborative informal networks is proposed to produce high quality products

Several products have become competitive in a variety of applications Gnome – desktop environments Linux – operating systems Mozilla – web browsers Apache – web servers

FLOSSFree/Libre and Open Source Software

Combination of the terms “Free Software” – Free Software Foundation “Open Source Software” – Open Source Initiative

Emphases the ‘libre’ meaning of the word free

Refers to software (available source code) that gives the freedom to the user to: run the program for any purpose to study the code to modify the code to distribute the original or modified program without any

restriction or any obligation

Free Software – Open Source Software

two ideological movements that target the same goal:

To evolve knowledge and develop better products

The motivations of each movement differ and many discussions have been made to disambiguate their nature

the terms “free software” and “open source software” are usually used interchangeably to describe the same thing

FLOSS Development

Collaboration and co-creation of the code by developers (professional & amateur programmers) and users of varying levels of programming experience

Typical FLOSS developers:

work:

• independently of any organisation

• for sake of the community projects

• isolated in their own pace, space and time

are:

• geographically dispersed

• simultaneously users of the software

Typical Developer Contribution

Start from existing or partial code

Take part in the development of a new tool or a utility

Post the evolving code to a newsgroup whose members will try out the program and provide feedback (programming solutions, improvements)

The enhanced code is then posted to the group, who accepts or discards it and redistribute the final product freely available to the wider community

One person = developer + user The developers are themselves users of the

software better understanding and solution of the problems or bugs

In Proprietary Software:

the developer has difficulties to understand the often ambiguous user requirements

Inadequate communication between the developer and the user creates inflexible to future changes products and products that are difficult to maintain

One person acting as developer and user simultaneously in the development of FLOSS products seems to be a prominent feature affecting the quality characteristics of the software product

FLOSS Quality The first FLOSS products were expert developers using

empirical procedures and metrics that ensure high quality products delivery

Today, the quality of FLOSS projects is not based on a specific method or metrics

The evaluation of quality is based on arbitrary criteria that usually are not sufficient or adequate to represent the quality of the product

Projects hosts provide limited metrics eg. sourceforge.net provides:

• projects size• number of developers• activity rank• weekly download

* http://www.infonomics.nl/FLOSS/report/

Free/Libre and Open Software: Survey and Study

International Institute of Infonomics University of Maastricht, The Netherlands in 2002

First attempt for quantification and measurement of underlying factors of quality

The Acronym FLOSS was used for first time

Delivered a large number of measurements and primary data related to FLOSS usage and development

The aims:

To provide indicators that rationally explain the functioning of the collaborative networks that do not have primary dependence on money

To investigate possible impact of non monetary collaborative development on computer industry, business and economy

ISO/IEC 9126 Quality Model A framework for the evaluation of software product

quality Applicable to every type of software Defines six high level product quality characteristics:

Portability Efficiency Reliability Functionality Usability Maintainability

Product Quality AttributesSub-characteristics

Measurable entity

Not defined in standard

The ISO/IEC 9126 Standard

ISO 9126 Divisions

The standard is divided into four parts which address:• the quality model - ISO/IEC 9126-1 (2001) • external metrics - ISO/IEC 9126-2 (2003) • internal metrics - ISO/IEC 9126-3 (2003) • quality in use metrics - ISO/IEC 9126-4 (2004)

Research methodology :

Survey

on-line questionnaire

http://sdragan.phpnet.us/questionnaire.htm

Department of Informatics Technological Educational Institute of Thessaloniki, Greece

Target group

Active persons of FLOSS projects, such as:

Project managers Programmers users

The research will be carried out

from April 2007 until August 2008

Questionnaire design

Php and MySql was used to implement the online questionnaire

21 questions relating to significant attributes describing human features and processes that have impact on quality of FLOSS according to ISO-9126

Emphasis on:

Simplicity, user friendliness and wide acceptance

unbiased understandable questions

Demographic questions related to the responders for

clustering of the results

investigation of possible impacts to the quality characteristics

Sample Issue Concerns Minimum tolerated sample size for smallest sub-group

of importance?

How to ensure a representative sample?

Our target group in the initial stage:

The Greek Floss community

Results may not be representative for the wider FLOSS community

Pilot Study: Questionnaire moderation & promotion

The initial questionnaire was

developed in the Greek language

distributed to many individuals in charge of Greek FLOSS communities aiming

• to evaluate the questionnaire regarding the validity and outline

• to secure future commitment from the FLOSS community

9 responses were received with useful comments

The feedback was used to improve the questionnaire

Questionnaire moderation & Promotion

The moderated questionnaire was translated to English in order to reach a greater population outside Greece

Available on-line to enable easy completion and analysis of the results

The on-line questionnaire has been promoted to many FLOSS communities in order to ensure statistically sufficient responses and produce valid results

Survey process

Acquisition of primary data that describe attributes and processes constituting FLOSS development

Data Storage in a database system in order to simplify appropriate transformation of the data before the analysis

Data analysis

Statistical analysis (SPSS-15)

Data mining methods will be applied to predict the behaviour of the project

• to classify projects according to their quality measurements

• to point out quality related attributes of the FLOSS development

Data Analysis

Considerable effort was made to enable the extraction of valid results rapidly from the primary data

A system that collects the questionnaire responses and processes them accordingly has been developed

Expected Outcome

to reveal factors that may have impact on the quality of FLOSS products

to increase the comprehension of FLOSS quality attributes

to raise awareness of the dynamics that influence the development process

to contribute to a more scientific, rational and structured assessment of FLOSS products & processes

Conclusions The initial stage of a research project aiming to

determine quality level of FLOSS products was described

Our approach evolved the research approaches that concentrate on internal characteristics of ISO-9126 to include external characteristics and quality in use

The correlation of quality characteristics of ISO-9126 with measurable aspects and underlying factors of FLOSS collaborative development was expected to provide a more scientific, rational and structured tool for FLOSS product assessment

Further Work Stage 1:

Extending the research group

• Collaboration welcome!

• Searching for EU funding

• Preparing proposal

Improving the questionnaire

Further promotion of the questionnaire

Further Work Stage 2:

Collecting data

Analysis of the collected data

• Statistical analysis of the data by SPSS

• Application of data mining methods

Stage 3:

Point out different perspectives and summarize them to useful information

Dissemination

Thank you for your attention

Any questions..