Quality is Everything Geoffrey Bessin Market Manager, Software Quality IBM Rational.
Post on 15-Jan-2016
217 Views
Preview:
Transcript
Quality is Everything
Geoffrey BessinMarket Manager, Software Quality
IBM Rational
2
3
What is “quality”?
FURPS+Functionality
UsabilityReliability
PerformanceSupportability
+ others
Acceptable GOAL RESULT
4
Software “engineering”
Software bugs cost the US economy an estimated $59.5
billion in 2002.1
On average, professional coders make 100 to 150 errors in every thousand lines of code they write.2
In 2002, canceled projects cost firms $67 billion; overruns on
other projects racked up another $21 billion.3
1 NIST2 Watts Humphrey, Carnegie Mellon3 Standish Group
5
Room for improvement
DEFECT COUNT
RESULTING QUALITY
Process Problems
•Waterfall-like development
•Poor asset management and measurement
Product Problems
•Lack of proactive validation techniques
•Data disconnect between IT Operations and Development
People Problems
•No shared responsibility for quality
•Over-reliance on test team
70% of organizati
ons are only CMM
Level 1
50% of total
project cost goes toward
software maintenan
ce25% of
projects are
cancelled; another 25% are
late
6
It’s the other person’s fault!
“….we estimate that 40 percent of unplanned application downtime is caused by application failures; 40 percent by application errors; and only 20 percent by hardware (server and network), environmental factors (heating, cooling and power failures) and disasters”
“70%-80% of project failures are tied to poor analysis”
7
If you’re thinking, “Not my problem” – then you’ve got a problem.
8
Solution Selling 101
SQUEAK!!
9
Solution Selling – Graduate Course
10
Is this problem related to hardware? Is an obvious application failure logged? Is this problem related to memory or performance issues of the software? Which additional data/traces should be captured to accelerate problem determination? What is the severity of the problem?
Is this problem related to hardware? Is an obvious application failure logged? Is this problem related to memory or performance issues of the software? Which additional data/traces should be captured to accelerate problem determination? What is the severity of the problem?
Operations
Is this problem a defect? What scenario reproduces the defect? What is the customer priority? Is this defect related to a specific configuration?
Is this problem a request for enhancement? What is the requirement or use case? What is the customer priority?
Is this problem a defect? What scenario reproduces the defect? What is the customer priority? Is this defect related to a specific configuration?
Is this problem a request for enhancement? What is the requirement or use case? What is the customer priority?
Tech Support
Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?
Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?
Tester
Deployment TransitionConstructionElaborationInceptionDeployment TransitionConstructionElaborationInception
PackagedApplication
RDBMS
WebService
Jservice
JSPs, struts,Servlets
Enterprise ACICS
Maintenance of Existing Systems and Software
App Servers
EJBs
App Servers
WebServer
EJBs
EdgeServer
Enterprise B
Traceability
Test Case
Test Case
Execution Trace
Test Case
Defect
Defect
Code
Code
11
Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?
Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?
Tester
What is the business priority of the defect? What is the impact on existing customers? How quickly is a fix needed? What is the relative importance of this defect compared to high-priority enhancements?
What is the business priority of the defect? What is the impact on existing customers? How quickly is a fix needed? What is the relative importance of this defect compared to high-priority enhancements?
Analyst
What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?
What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?
Architect
InceptionDeployment TransitionConstructionElaboration
PackagedApplication
RDBMS
WebService
Jservice
JSPs, struts,Servlets
Enterprise ACICS
Maintenance of Existing Systems and Software
App Servers
EJBs
App Servers
WebServer
EJBs
EdgeServer
Enterprise B
Traceability
Test Case
Test Case
FEATreq
FEATreqArchitecture
Code
Code
Execution Trace
Test Case
Defect
Defect
Planning
12
What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?
What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?
Architect
Which components are suspect? Are there any clues in the captured traces? Is this defect reproducible in the development or test environment? Is there a need for additional traces?
What is the foreseeable impact and risk? What other components are impacted?How much time is needed to fix the issue?
Which components are suspect? Are there any clues in the captured traces? Is this defect reproducible in the development or test environment? Is there a need for additional traces?
What is the foreseeable impact and risk? What other components are impacted?How much time is needed to fix the issue?
Developer
Is there a significant architectural impact on the system? How many components are impacted? Is there a need to refactor the architecture?
Is there a significant architectural impact on the system? How many components are impacted? Is there a need to refactor the architecture?
Architect
How much regression testing is required? Which use cases are impacted? Which components are impacted? Which tests need to be re-executed?
What is the impact on existing plan? How many resources need to be engaged? Is it possible to stay on original schedule?
How much regression testing is required? Which use cases are impacted? Which components are impacted? Which tests need to be re-executed?
What is the impact on existing plan? How many resources need to be engaged? Is it possible to stay on original schedule?
Tester
What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?
What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?
Analyst
ElaborationTriageDeployment TransitionConstruction
PackagedApplication
RDBMS
WebService
Jservice
JSPs, struts,Servlets
Enterprise ACICS
Maintenance of Existing Systems and Software
App Servers
EJBs
App Servers
WebServer
EJBs
EdgeServer
Enterprise B
Traceability
Architecture
Test Case
Test Case
FEATreq
FEATreq
Code
Code
Execution Trace
Test Case
Defect
Defect
Planning
Driver
13
What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?
What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?
Analyst
What is the impact of these changes to existing components?
Which components are impacted by the changes? Is there any regression on these components?
What is the impact of these changes to existing components?
Which components are impacted by the changes? Is there any regression on these components?
Developer
What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?
What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?
What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?
What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?
Tester
ConstructionAnalysisTriageDeployment Transition
PackagedApplication
RDBMS
WebService
Jservice
JSPs, struts,Servlets
Enterprise ACICS
Maintenance of Existing Systems and Software
App Servers
EJBs
App Servers
WebServer
EJBs
EdgeServer
Enterprise B
Traceability
Architecture
Test Case
Test Case
FEATreq
FEATreq
Code
Code
Execution Trace
Test Case
Defect
Defect
Planning
14
What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?
What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?
What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?
What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?
Tester
Is the fix delivery vehicle working? Does the install work? Does the fix solve the issue?
Is the fix delivery vehicle working? Does the install work? Does the fix solve the issue?Tester
Does this solve the issue? Does this solve the issue?
Operations
Is the customer satisfied with the fix? Is the customer satisfied with the fix?
Tech Support
TransitionFixAnalysisTriageDeployment
PackagedApplication
RDBMS
WebService
Jservice
JSPs, struts,Servlets
Enterprise ACICS
Maintenance of Existing Systems and Software
App Servers
EJBs
App Servers
WebServer
EJBs
EdgeServer
Enterprise B
Traceability
Architecture
Test Case
Test Case
FEATreq
FEATreq
Code
Code
Execution Trace
Test Case
Defect
Defect
Planning
Driver
15
Don’t be intimidated
16
The IBM Software Development Platform
Analyst Architect Developer Tester
Visually construct, transform,integrate
and generate code
Design, create,
and execute tests
Model, simulate, assemble,
and monitor business
processes
Modelapplications
and data
IndustryStandards
Training & Mentoring
Microsoft
IBM and3rd Party Servers
DB2LotusRationalTivoliWebSphere
Pervasiveand
Embedded
Partners
Follow a common process Track project status Manage requirements
Manage changeand assets
Manage qualityProject Manager
Deployer
Provision, configure, tune and
troubleshoot applications
Develop Iteratively
Focus on Architecture
Manage Changeand Assets
Continuously EnsureQuality
17
Building a quality ecosystem for flexibility and growth
Role-customized software quality
tools and processes
Global deployment awareness
Seamless asset unification
Open, extensible, technology/platform-
independent infrastructure
Execution Trace
UCreq
FEATreq
Architecture
Code
UC Realizations
Test CaseTest Plan
H
H
H
H
HYADES
Analys
t
Teste
r
Architect
Developer
IT Operations
Verify and validate from the beginning
Maintain traceable, actionable cross-lifecycle
linkages
Incorporate post-deployment analysis
18
Application Lifecycle ManagementClosed-loop problem determination, repair, and redeployment
Reduce business downtimewhile accelerating business value
throughput…
1. Quickly discover and understand application-level errors even after deployment
2. Speed Tivoli-aware application fix and (re)build
3. Optimize and accelerate (re)deployment
…by bridging developmentand operations teams
Sample TechnologiesSample Technologies Tivoli Monitoring Tivoli Configuration Management Tivoli Provisioning/Orchestration Rational Test and Runtime Analysis Rational Version and Build Control Rational Modeling
Monitor-readyApplication
Development
Monitor-readyApplication
Development
ProblemDetermination
ProblemDetermination
Operations TeamOperations Team
DevelopmentTeam
DevelopmentTeam
1122
33 Rapid DeploymentRapid Deployment
19
Quality is FREE
- A well-defined process with unanimous buy-in from team members- Focused attention to value-add, detail and quality across the lifecycle- Constant evaluation of end-user feedback to ensure reinvigoration of the product line and satisfied, repeat customers
- Poorly defined process with inconsistent buy-in - Front-loaded requirements gathering without reassessment or validation- Absent or limited architectural and code quality assurance - Late cycle system testing - Limited post-deployment monitoring and assessment
20
Quality enables differentiation
Through 2006, 80 percent of development organizations will not follow strict software
engineering practices and will fail in 70 percent of their application delivery efforts
21
Quality enables innovation
22
An IBM & Gartner “rule of thumb” for the relative costs to fix defects: Requirements 1.5 Design: 1.0 Construct: 1.5 Prior to test: 10.0 Testing: 60.0 Production: 100.0
It’s your choice
Cost of a Defect (when identified)
0
20
40
60
80
100
120
Requir
emen
ts
Design
Constr
uct
Prior t
o te
st
Testin
g
Produ
ction
Development Phase
Re
lati
ve
Co
st
Cost of a Defect
Note: Does not include business implications of lost revenue and long term impact to business image
You can wait…..
23
….or you can act
Take personal responsibilityWork together
Establish a right-sized, iterative processAutomate whenever, whereverNever assume and don’t settle
24
top related