Software quality

Software Quality

In practice Quality of a system can be a vague undefined attributeWe therefore need to define precisely what qualities we require of a systemFor any software system there should be three specifications

bull A functional specification describing what the system is to dobull A quality (or attribute) specification concerned with how well the functions are to operatebull A resource specification concerned with how much is to be spent on the system

Quality FactorsTo identify specific product qualities that are appropriate to software James A McCall grouped software qualities into three sets of quality factors

bull Product operation quality factorsbull Correctness Traceability consistency completenessbull Reliability Error tolerance consistency accuracybull Efficiency Execution and storage efficiencybull Integrity Access control access auditbull Usability Operability training communicativeness Input Output volume

Input Output ratebull Product revision quality factors

bull Maintainability Consistency simplicity modularity self-descriptivenessbull Testability Simplicity modularity instrumentation self-descriptivenessbull Flexibility Modularity generality expandability self-descriptiveness

bull Product transition quality factorsbull Portability Modularity self-descriptiveness machine independence

software system independencebull Reusability Generality modularity software system independence machine

independence self-descriptivenessbull Interoperability Modularity communications commonality data

commonality

ISO 9126This standard was published to tackle the question of the definition of software qualityISO 9126 identifies six software quality characteristics

bull Functionality suitability accuracy interoperability (interaction with other systems) compliance security

bull Reliability maturity (in terms of failure in system) fault tolerance recoverabilitybull Usability understandability learnability operabilitybull Efficiency time behavior resource behaviorbull Maintainability analyzability changeability stability testabilitybull Portability adaptability installability conformance replaceability

Techniques to help enhance Software Qualitybull Increasing visibilitybull Procedural structurebull Checking immediate stages through inspections

Process and Product QualityThe quality of a developed product is influenced by the quality of the production process This is important in software development as some product quality attributes are hard to assess However there is a very complex and poorly understood relationship between software processes and product quality

Process-based qualitybull There is a straightforward link between process and product in manufactured goodsbull More complex for software because

ndash The application of individual skills and experience is particularly important in software development

ndash External factors such as the novelty of an application or the need for an accelerated development schedule may impair product quality

bull Care must be taken not to impose inappropriate process standards - these could reduce rather than improve the product quality

Quality management activitiesbull Quality assurance

ndash Establish organisational procedures and standards for qualitybull Quality planning

ndash Select applicable procedures and standards for a particular project and modify these as required

bull Quality controlndash Ensure that procedures and standards are followed by the software development

teambull Quality management should be separate from project management to ensure

independence

Software quality assurance (SQA) consists of a means of monitoring the software engineering processes and methods used to ensure quality It does this by means of audits of the quality management system under which the software system is created These audits are backed by one or more standards usually ISO 9000 or CMMITypical quality assurance activities includeEstablishing a quality management systemDefining processes

Training personnelEstablishing mechanisms for quality assuranceProviding guidance to project managersEffecting continuous improvement

Software Quality Assurance encompasses the entire software development process which includes processes such as software design coding source code control code reviews change management configuration management and release management Whereas software quality control is a control of products software quality assurance is a control of processesIt is distinct from software quality control which includes reviewing requirements documents and software testing

Cost of quality can be divided into the following groupsbull Preventive costbull Detection costbull Internal failure costbull External failure cost

bull Start-up costsbull Surveybull Policybull Standardsbull Metrics identificationbull Tools

bull Project related costsbull Quality planbull Quality assurance supportbull Supervision costbull Quality control costs eg reviews testing

bull Continuing costsbull Managementbull Staff trainingbull Standards maintenancebull Researchbull Metrics maintenance

Quality assurance benefitsbull Quantitative

bull Reduced costsbull Greater efficiencybull Better performancebull Less unplanned workbull Fewer disputes

bull Qualitativebull Improved visibility and predictabilitybull Reduced riskbull Problems show up earlierbull Better qualitybull Improved customer confidencebull Portable and reusable product

bull Better control over contracted products

QUALITY PLANSAs per IEEE a quality plan might have entries for

bull Purpose scope of planbull List of references to other documentsbull Management including organization tasks and responsibilitiesbull Documentation to be producedbull Standard practices and conventionsbull Reviews and auditsbull Testing and configuration managementbull Problem reporting and corrective actionbull Tools techniques and methodologiesbull Code media and supplier controlbull Records collection maintenance and retentionbull Trainingbull Risk management

Quality Review involves checking the software development process to ensure that procedures and standards are being followed There are two approaches to quality control

ndash Quality reviewsA group of people carefully examine part or all of a software system and its associated documentation Code designs specifications test plans standards etc can all be reviewed Software or documents may be signed off at a review which signifies that progress to the next development stage has been approved by management

bull Review functions Quality function - they are part of the general quality management processProject management function - they provide information for project managers Training and communication function - product knowledge is passed between development team members

bull Review resultsComments made during the review should be classifiedNo action No change to the software or documentation is requiredRefer for repair Designer or programmer should correct an identified faultReconsider overall design The problem identified in the review impacts other parts of the design Some overall judgement must be made about the most cost-effective way of solving the problemRequirements and specification errors may have to be referred to the client

ndash Automated software assessment and software measurement

Software Quality Management SystemA quality management system (quality system) is the principal methodology used by organizations to ensure that the products they develop have the desired quality It consists of the following

bull Managerial structure and individual responsibilities It is the responsibility of the organization as a whole It should have the support of the top management

bull Quality system activities includebull Auditing of the projectsbull Review of the quality systembull Development of standardsbull Production of reports for top management

Quality Assurance and Standardsbull Standards are the key to effective quality managementbull They may be international national and organizational or project standardsbull Product standards define characteristics that all components should exhibit eg a common

programming stylebull Process standards define how the software process should be enacted

Importance of Standardsbull Encapsulation of best practice- avoids repetition of past mistakesbull They are a framework for quality assurance processes - they involve checking compliance to

standardsbull They provide continuity - new staff can understand the organisation by understanding the

standards that are used

ISO 9000International standards such as ISO 9000 provide guidance on how to organize and maintain a quality systemIt specifies a set of guidelines for repeatable and high quality product developmentISO 9000 and ISO 9003 apply to software organizations

Benefits of ISO 9000 Certificationbull Confidence of customersbull Well-documented softwarebull Makes the process focused efficient and cost-effectivebull Weak points of the organization are identified and remedial action implementedbull Sets the basic framework for total quality management

How to get ISO 9000 certificationbull Applicationbull Pre-assessmentbull Documents review and adequacy of auditsbull Compliance auditbull Registrationbull Continued surveillance

Summary of ISO 9001 Requirements

bull Management responsibilitybull Quality Systembull Contract Reviewsbull Design Controlbull Document controlbull Purchasingbull Purchaser Supplied productbull Product Identificationbull Process controlbull Inspection and testingbull Inspection Measuring and Test Equipmentbull Inspection and Test Statusbull Control of Nonconforming Productbull Corrective actionbull Handlingbull Quality recordsbull Quality Auditsbull Trainingbull Servicingbull Statistical Techniques

Contract Review ISO 9001 requires that contracts be reviewed to determine whether the requirements are adequately defined agreed with the bid and can be implementedDesign Control It requires that procedures to control and verify the design be established This includes planning design activities identifying inputs and outputs verifying the design and controlling design changesDocument Control The distribution and modification of documents be controlledPurchasing The purchased products should confirm to their specified requirements This includes assessment of potential sub-contractors and verification of purchased productsPurchaser-Supplied product It requires that any purchaser-supplied material be verified and maintained Product Identification and Traceability It requires that the product be identified and traceable during all stages of production delivery and installationProcess Control It requires that production processes be defined and planned This includes carrying out production under controlled conditions according to documented instructions Special processes that cannot be fully verified are continuously monitored and controlledInspection and Testing It requires that incoming materials be inspected or certified before use and that in-process inspection and testing be performed Final inspection and testing are performed prior to release of finished product Records of inspection and testing are kept Inspection Measuring and Test Equipment It requires that equipment used to demonstrate conformance be controlled calibrated and maintained When test hardware or software be used it is checked before use and rechecked at prescribed intervals

Inspection and Test Status It requires that the status of inspections and tests be maintained for items as they progress through various processing steps Control of nonconforming product It requires that nonconforming product be controlled to prevent inadvertent use or installationCorrective action It requires that the causes of nonconforming product be identified Potential causes of nonconforming product are eliminated procedures are changed resulting from corrective actionHandling Storage Packaging and Delivery It requires that procedures for handling storage packaging and delivery be established and maintained Quality records It requires that quality records be collected maintained and dispositionedInternal Quality Audits It requires that audits be planned and performed The results of audits are communicated to management and any deficiencies found are correctedTraining It requires that training needs be identified and that training be provided since selected tasks may require qualified personnel Records of training are maintainedServicing It requires that servicing activities be performed as specified Statistical Techniques It states that appropriate adequate statistical techniques are identified and should be used to verify the acceptability of process capability and product characteristics

Shortcomings of ISO 9000 Certification It requires a software production process to be adhered to but does not guarantee the

process to be of high quality No international accreditation agency exists Organizations getting ISO 9000 certification often tend to downplay domain expertise ISO 9000 does not automatically lead to continuous process improvement

Capability Maturity ModelIt classifies software development industry into the following five maturity models

1 Initial characterized by ad hoc activities2 Repeatable Management practices like tracking costs and schedule are established3 Defined The processes for both management and development activities are defined and

documented The processes though defined product qualities are not measured4 Managed Product and process metrics are measured and used for evaluating project

performance rather than improving the process5 Optimizing Continuous process improvement is achieved both by carefully analyzing the

quantitative feedback from process measurements and from application of innovative ideas and technologies

Comparison between ISO 9000 and CMM

bull ISO 9000 can be quoted in official documents for communication with external parties

bull ISO 9000 applies to a range of organizations

bull ISO aims at level 3 of CMM

bull States important standards which all organizations should maintain

bull Product based

bull CMM is for internal purposes

bull Applies specially to Software industry

bull CMM aims to achieve Total Quality management

bull Provides a list of Key process areas (KPAs) on which the organization at any maturity level needs to concentrate

bull Process based

Six SigmaSix sigma at many organizations simply means striving for near perfection To achieve six sigma a process must not produce more than the acceptable defectsThe fundamental object of six sigma methodology is the implementation of a measurement-based strategy that focuses on process improvement and variation reduction through the application of six sigma improvement projects

What is SigmaSigma Level Defects Per Million Rate of

Opportunities Improvement

1 s 690000 2 s 308000 2 times 3 s 66800 5 times 4 s 6210 11 times 5 s 230 27 times 6 s 34 68 times

It has three core stepsbull Define customer requirements deliverables and project goals via well-defined methods of

customer communicationbull Measure the existing process and its output to determine current quality performance

(collect defect metrics)bull Analyze defect metrics and determine the vital few causes

It also suggests additional steps for improvementbull Improve the process by eliminating the root cause of defectsbull Control the process to ensure that future work does not reintroduce the causes of defects

Six Sigma Project Methodology

Applying Six Sigma to software development makes product development and other projects transparent to both management and customers Transparency requires an important cultural change As a result after transparency is achieved completing accurate project estimations while meeting both deadlines and customer requirements becomes a lot easier

There is no SW metrics that serves all It depends what kind of software we are developing or installing Finding good metrics for software development or deployment is a major task in itself Software development must adapt to the ever changing human ideas beliefs fears and the corresponding environment We cannot restrict ourselves to software engineering we need focus on project management to improve both software development and deployment

And we need measurements in order to know that we implemented the targets we had planned for

Principle No 11048759 Measure customer related metrics only

o Use Combinatory Metrics to cover all topicsPrinciple No 2

1048759 Adjust to moving targetso Your goals may need change accept change and manage it accordingly

Principle No 31048759 Enforce measurement

o Do not enforce meeting targets

What Makes Six Sigma Differentbull Versatilebull Breakthrough improvementsbull Financial results focus

bull Process focusbull Structured amp disciplined problem solving methodology using scientific tools and techniquesbull Customer centeredbull Involvement of leadership is mandatorybull Training is mandatory bull Action learning (25 class room 75 application)bull Creating a dedicated organisation for problem solving (8550 Rule)

Benefits of Six Sigmabull Generates sustained successbull Sets performance goal for everyonebull Enhances value for customersbull Accelerates rate of improvementbull Promotes learning across boundariesbull Executes strategic change

Software AuditSoftware audit is a systematic and independent examination to determine the following

Are quality related arrangements implemented as plannedAre results consistent with expectations

Adequacy Audit determines the extent to which documented QMS conforms to applicable standardsCompliance Audit is aimed at determining the extent to which the implemented system conforms to documented systemsSurveillance Audit ensures continued adherence to standards and continuous improvement

First party auditsPerformed in-houseQMS requirementEnsures effective implementationEvaluate any particular quality problem

Second party auditsPerformed by clientsVendor selection

Third party auditsCertificationSurveys

Documentation standardsParticularly important - documents are the tangible manifestation of the softwareDocumentation process standards

Concerned with how documents should be developed validated and maintainedDocument standards

Concerned with document contents structure and appearance

Document interchange standardsConcerned with the compatibility of electronic documents

bull Document identification standardsndash How documents are uniquely identified

bull Document structure standardsndash Standard structure for project documents

bull Document presentation standardsndash Define fonts and styles use of logos etc

bull Document update standardsndash Define how changes from previous versions are reflected in a document

bull Interchange standards allow electronic documents to be exchanged mailed etc

bull Documents are produced using different systems and on different computers Even when standard tools are used standards are needed to define conventions for their use eg use of style sheets and macros

bull Need for archiving The lifetime of word processing systems may be much less than the lifetime of the software being documented An archiving standard may be defined to ensure that the document can be accessed in future