www.eu-etics.org INFSOM-RI-026753 A Quality Certification A Quality Certification Model for Grid Research Model for Grid Research Projects Projects the ETICS feasibility the ETICS feasibility Study Study Adriano Rippa Adriano Rippa ( [email protected][email protected]) - Engineering Ingegneria Informatica s.p.a. - Engineering Ingegneria Informatica s.p.a. On behalf of the ETICS Project On behalf of the ETICS Project
22
Embed
A Quality Certification Model for Grid Research Projects the ETICS feasibility Study
A Quality Certification Model for Grid Research Projects the ETICS feasibility Study. Adriano Rippa ( [email protected] ) - Engineering Ingegneria Informatica s.p.a. On behalf of the ETICS Project. Summary. Grid Quality Certification Model (GQCM) - UPDATE - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
www.eu-etics.org
INFSOM-RI-026753
A Quality Certification Model for A Quality Certification Model for Grid Research ProjectsGrid Research Projects
the ETICS feasibility Studythe ETICS feasibility Study
Adriano RippaAdriano Rippa(([email protected]@eng.it)) - Engineering Ingegneria Informatica s.p.a.- Engineering Ingegneria Informatica s.p.a.On behalf of the ETICS ProjectOn behalf of the ETICS Project
ETICS All Hands Meeting - Geneva 21-23/11/2007 2INFSOM-RI-026753
• Comparing GQCM and other QA standardsComparing GQCM and other QA standards
• TimelineTimeline
• Gotten feedbacksGotten feedbacks
• PublicationsPublications
• A possible first implementationA possible first implementation
ETICS All Hands Meeting - Geneva 21-23/11/2007 3INFSOM-RI-026753
In Madison we left here…
• The Grid Quality Assurance The Grid Quality Assurance Certification Process Certification Process (GQACP) was organized in (GQACP) was organized in three perspectives looking three perspectives looking at Code, Platforms and at Code, Platforms and Standards quality aspects.Standards quality aspects.
• Now it asses the quality Now it asses the quality against the same macro-against the same macro-aspects but…aspects but…
CodeCode
PlatformsPlatforms
StandardsStandards
ETICS All Hands Meeting - Geneva 21-23/11/2007 4INFSOM-RI-026753
…Now we are here
• GQCM GQCM has been reorganized as follows:has been reorganized as follows:
– A new name!A new name!
– GQCMGQCM has been described has been described according to several ISO standardsaccording to several ISO standards (e.g. ISO/IEC (e.g. ISO/IEC 25000, 1459825000, 14598))
– GQCMGQCM has been restructured has been restructured according to ISO 25041according to ISO 25041
– Quality attributesQuality attributes has been named using the same terminology has been named using the same terminology as as ISO 9126ISO 9126
– More metrics More metrics has been added including bounds and target has been added including bounds and target valuesvalues
ETICS All Hands Meeting - Geneva 21-23/11/2007 5INFSOM-RI-026753
GQCM: the structure
• GQCM is structured in GQCM is structured in Evaluation Modules (EM)Evaluation Modules (EM) according to ISO 25041according to ISO 25041
• The set of evaluation techniques are grouped in The set of evaluation techniques are grouped in families. Every family is an EMfamilies. Every family is an EM
– Technique:Technique:– Structural testing. The intention of this EM is to test specific classes Structural testing. The intention of this EM is to test specific classes
that are identified by static measures as being statistically more that are identified by static measures as being statistically more likely to have many errors. likely to have many errors.
– Applicability:Applicability:– Object oriented programming languagesObject oriented programming languages
– Input:Input:– source codesource code
ETICS All Hands Meeting - Geneva 21-23/11/2007 9INFSOM-RI-026753
Evaluation Modules 4/5
• EM Functional testingEM Functional testing– Quality characteristics:Quality characteristics:
– The purpose of this EM is both to check platform compliance and The purpose of this EM is both to check platform compliance and to check to functional abilities of the software product.to check to functional abilities of the software product.
– Applicability:Applicability:– GeneralGeneral
– Input:Input:– compiled codecompiled code– user documentationuser documentation
ETICS All Hands Meeting - Geneva 21-23/11/2007 10INFSOM-RI-026753
Evaluation Modules 5/5
• EM Standards complianceEM Standards compliance– Quality characteristics:Quality characteristics:
– This EM has a good potential for automating the test. This EM has a good potential for automating the test.
– Applicability:Applicability:– Standard specificStandard specific
– Input:Input:– compiled codecompiled code
ETICS All Hands Meeting - Geneva 21-23/11/2007 11INFSOM-RI-026753
GQCM: Final Score
• Final score should be provided according to the following Final score should be provided according to the following schema. The items which should be available for the users are:schema. The items which should be available for the users are:– A A table summarizing the resultstable summarizing the results– A A list of passed and non passed testslist of passed and non passed tests– All the All the important informationimportant information as: as:
– Standard compliance (for each standard: name, version, date)Standard compliance (for each standard: name, version, date)
– Identification of evaluation report (organization, report number, date)Identification of evaluation report (organization, report number, date)
– Identification of certification body (organization, contact information)Identification of certification body (organization, contact information)
– Certification data (dates, certification number)Certification data (dates, certification number)
– Electronic signature of certification record Electronic signature of certification record
ETICS All Hands Meeting - Geneva 21-23/11/2007 12INFSOM-RI-026753
GQCM: Summarizing Table example
Consolidated eval. result
EM Static
EM Coding
EM structural
EM Functional
EM Std compliance
EM eval. result M M G G G Y
Functionality G G G
Accuracy G G G
Interoperability G G
Compliance Y (Y)
Reliability M M G G
Maturity M M G G
Maintainability M M G
Analyzability M M G
Changeability M M
Testability M M
Portability G G
Adaptability G G
Installability G G
G = GoodM = MediumP = Poor
ETICS All Hands Meeting - Geneva 21-23/11/2007 13INFSOM-RI-026753
GQCM integration in an ISO 9126 certified organization
• Facts:Facts:– ISO 9126 define quality attributes ISO 9126 define quality attributes – GQCM support the measure and evaluation of quality metricsGQCM support the measure and evaluation of quality metrics
• Why they are compatible each other?Why they are compatible each other?– Many ISO9126 aspects can be evaluated by GQCM Many ISO9126 aspects can be evaluated by GQCM GQCM GQCM
apply what ISO 9126 asksapply what ISO 9126 asks– Any tool implementing GQCM and can be integrated as Any tool implementing GQCM and can be integrated as
• SoSo– GQCM can be both an independent quality model and a “tool” to GQCM can be both an independent quality model and a “tool” to
be easily compliant with ISO 9126be easily compliant with ISO 9126
ETICS All Hands Meeting - Geneva 21-23/11/2007 14INFSOM-RI-026753
GQCM integration in a CMMi certified organization
• FactsFacts– CMM is process oriented (it gives guidelines).CMM is process oriented (it gives guidelines).– GQCM is product oriented.GQCM is product oriented.
• Hypothesis:Hypothesis:– CMM2 CMM2
– asks to “control and value”asks to “control and value”– GQCM proposes the measures and metrics to be collected, hence GQCM proposes the measures and metrics to be collected, hence
controlling and evaluating specific sw related itemscontrolling and evaluating specific sw related items
– CMM3 CMM3 – asks to “use internal standards and have a reference model”asks to “use internal standards and have a reference model”– GQCM proposes code and implementation conventions (how to GQCM proposes code and implementation conventions (how to
organise and structure code, how to define tests, etc.)organise and structure code, how to define tests, etc.)
ETICS All Hands Meeting - Geneva 21-23/11/2007 15INFSOM-RI-026753
GQCM integration in a CMM certified organization
• CMM4 CMM4 – asks to “have measured and controlled processes using quantitative asks to “have measured and controlled processes using quantitative
and statistical techniques”and statistical techniques”– GQCM proposes the continuous collection of quality data to perform GQCM proposes the continuous collection of quality data to perform
trends analysis.trends analysis.
• CMM5 CMM5 – asks to have an “improvement process and quality based on asks to have an “improvement process and quality based on
measurement”measurement”– GQCM provides numerical reference to set the improvement GQCM provides numerical reference to set the improvement
processes.processes.
– Our ThesisOur Thesis– GQCM is not in contrast with CMMI and can be integrated in GQCM is not in contrast with CMMI and can be integrated in
organization as a tool to support CMMI adoptionorganization as a tool to support CMMI adoption
ETICS All Hands Meeting - Geneva 21-23/11/2007 16INFSOM-RI-026753
The timeline
OCTOBEROCTOBER DECEMBERDECEMBER
ECHOGRID/EUChinagridECHOGRID/EUChinagrid
Conference 24-25 April Conference 24-25 April
BeijingBeijing
MAYMAYFEBRUARYFEBRUARY
OGF 20/EGEE UFOGF 20/EGEE UF
7-11 May 7-11 May
ManchesterManchester
Belief ConferenceBelief Conference
25-28 June 25-28 June
Rio de JaneiroRio de Janeiro
OGF 21OGF 21
15-19 October15-19 October
SeattleSeattle
QUALIPSO QUALIPSO
ConferenceConference
16-17 January 200816-17 January 2008
RomeRome
NowNow
EGEE ’07EGEE ’07
1-5 October1-5 October
BudapestBudapest
EELA 3° EELA 3° Conference Conference
3-5 December3-5 December
CataniaCatania
AHMAHM
23-25 May 23-25 May
MadisonMadison
ETICS All Hands Meeting - Geneva 21-23/11/2007 17INFSOM-RI-026753
• When/WhereWhen/Where• EELA third conferenceEELA third conference
• 3-5 December 20073-5 December 2007
• Catania/ItalyCatania/Italy
• QUALIPSO first conferenceQUALIPSO first conference• 16-17 January 200816-17 January 2008
• Rome/ItalyRome/Italy
• The first The first quality oriented conferencequality oriented conference!!
ETICS All Hands Meeting - Geneva 21-23/11/2007 18INFSOM-RI-026753
GQCM: feedbacks
• The model has got good feedbacks and The model has got good feedbacks and acknowledgementsacknowledgements
• People are asking for more informationPeople are asking for more information
• Some issues about the name of the model (solved)Some issues about the name of the model (solved)
• One issue about ISO9000 compliance (outside the One issue about ISO9000 compliance (outside the scope?)scope?)
• Requests for metrics about the process Requests for metrics about the process outside outside the scope?the scope?
ETICS All Hands Meeting - Geneva 21-23/11/2007 19INFSOM-RI-026753
GQCM: future and open issues
• Within ETICS 2Within ETICS 2• ImplementationImplementation• ImprovementsImprovements• Pilot projects to test itPilot projects to test it• ISO standardisationISO standardisation
• Open issueOpen issue• First implementation?First implementation?
ETICS All Hands Meeting - Geneva 21-23/11/2007 20INFSOM-RI-026753
What we have now?
• Programming languages Programming languages Java/Phyton Java/Phyton– Others?Others?
• JDepend can calculateJDepend can calculate– Number of classes indicator of the extensibility of the packageNumber of classes indicator of the extensibility of the package– Afferent Couplings indicator of the package’s responsibilityAfferent Couplings indicator of the package’s responsibility– Efferent Couplings … independenceEfferent Couplings … independence– Abstractness 0/1 Abstractness 0/1 package concrete – abstract package concrete – abstract– Instability 0/1 Instability 0/1 – Distance Distance indicator of the package bilance between abstractness and stability indicator of the package bilance between abstractness and stability
• PMD scans Java source code and looks for potential problems like:PMD scans Java source code and looks for potential problems like:– Possible bugs - empty try/catch/finally/switch statementsPossible bugs - empty try/catch/finally/switch statements– Dead code - unused local variables, parameters and private methodsDead code - unused local variables, parameters and private methods– Suboptimal code - wasteful String/StringBuffer usageSuboptimal code - wasteful String/StringBuffer usage– Overcomplicated expressions - unnecessary if statements, for loops that could be while loopsOvercomplicated expressions - unnecessary if statements, for loops that could be while loops– Duplicate code - copied/pasted code means copied/pasted bugsDuplicate code - copied/pasted code means copied/pasted bugs
ETICS All Hands Meeting - Geneva 21-23/11/2007 21INFSOM-RI-026753
A possible first “implementation”?
• We can implement partially three evaluation modulesWe can implement partially three evaluation modules– EM: Static Analysis (Only one measure..)EM: Static Analysis (Only one measure..)
– WMC: looks at the complexity of classesWMC: looks at the complexity of classes
– EM: Coding StyleEM: Coding Style– Only some measurement but we can have a realistic value from itOnly some measurement but we can have a realistic value from it
– EM: Structural TestingEM: Structural Testing – it is expected that a small part of the code contain a relative large it is expected that a small part of the code contain a relative large
number of the defects in the code. Some metrics are used to number of the defects in the code. Some metrics are used to identify potential problematic classes.identify potential problematic classes.
– Use other test cases than already used for statement coverage.Use other test cases than already used for statement coverage.
– The first implementation will aim at the programming languages The first implementation will aim at the programming languages Java and Python. […]Java and Python. […]
ETICS All Hands Meeting - Geneva 21-23/11/2007 22INFSOM-RI-026753