Page 1
12/12/2014
1
Data Analytics for Software Product Innovation
Guenther Ruhe
University of Calgary
©Guenther Ruhe
AGENDAAGENDA
How it began: The Esprit Project PROFES How it began: The Esprit Project PROFES
Product Innovation Product Innovation
Analytical Open InnovationAnalytical Open Innovation
AOI for Innovative ProductsAOI for Innovative Products
The Road AheadThe Road Ahead
PROFES 2014, Helsinki, Finland 2
Page 2
12/12/2014
2
©Guenther Ruhe
Participating Companies
PROFES 2014, Helsinki, Finland 3
©Guenther Ruhe
Project Team
PROFES 2014, Helsinki, Finland 4
Frank van Latum
Pasi Kuvaya Janne JarvinenenMarkku Oivo
Dietmar Pfahl Rini van Solingen Guenther Ruhe
Andreas Birk
Page 3
12/12/2014
3
©Guenther Ruhe
Elements of PROFES
• Combining and enhancing the strengths of goal‐oriented measurement, process assessment, product and process modelling and experience factory
PROFES 2014, Helsinki, Finland 5
ISO15504
GQMGQMISO
15504
PROFES
QIP/EFISO9126
©Guenther Ruhe
Focus on PPDs
• Focus on investigating the relationship between product and process quality
PROFES 2014, Helsinki, Finland 6
PPDPRODUCT PROCESS
Page 4
12/12/2014
4
©Guenther Ruhe
Different Facets of PPDs
PROFES 2014, Helsinki, Finland 7
Technologies Used
Design Inspections
ProductQuality
SoftwareProcess
Reliability Software Design
Low or AverageOverall Time Pressure
Context Characteristics
©Guenther Ruhe
AGENDAAGENDA
How it began: The Esprit Project PROFES How it began: The Esprit Project PROFES
Product Innovation Product Innovation
Analytical Open InnovationAnalytical Open Innovation
Analytics Case StudiesAnalytics Case Studies
The Road AheadThe Road Ahead
PROFES 2014, Helsinki, Finland 8
Page 5
12/12/2014
5
©Guenther Ruhe
Innovation – What is it … after all?
• Innovativeness is the measure of “newness”
• New to the:
World
Market
Industry
Adopting unit
Consumer
PROFES 2014, Helsinki, Finland 9
©Guenther RuhePROFES 2014, Helsinki, Finland 10
Crossing the Chasm
Page 6
12/12/2014
6
©Guenther Ruhe
Being New … Being First
PROFES 2014, Helsinki, Finland 11
• New technology
• New product line
• New product features
• New product design
• New process
• New service
• New customers
• New uses
• New quality
• New type of benefit
©Guenther Ruhe
A Powerful Force for Everyday Fitness
PROFES 2014, Helsinki, Finland 12
Page 7
12/12/2014
7
©Guenther Ruhe
AGENDAAGENDA
How it began: The Esprit Project PROFES How it began: The Esprit Project PROFES
Product Innovation Product Innovation
Analytical Open InnovationAnalytical Open Innovation
Analytics Case StudiesAnalytics Case Studies
The Road AheadThe Road Ahead
PROFES 2014, Helsinki, Finland 13
©Guenther Ruhe
Responding to change for gaining competitive advantage in the era of smart decisions will be based not on "gut instinct," but on predictive analytics.
PROFES 2014, Helsinki, Finland 14
Ginni Rometty, Chairman, President and CEO, IBM, 2013
Page 8
12/12/2014
8
©Guenther RuhePROFES 2014, Helsinki, Finland 15
Innovative product development: New ideas from leveraging external knowledge and resources, applying
innovative processes and technologies
©Guenther Ruhe
Open Innovation
• An (open) approach for integration of internal and external ideas and paths to market that merges distributed knowledge and ideas into production processes.
PROFES 2014, Helsinki, Finland 16
Chesbrough, H., “Open Innovation: The New
Imperative for Creating and Profiting
from Technology”, Harvard Business
Press, 2003.
Page 9
12/12/2014
9
©Guenther Ruhe
Open Innovation for New Products
PROFES 2014, Helsinki, Finland 17
©Guenther Ruhe
Analytic Open Innovation
• Open innovation utilizing the power of analytics (processes, tools, knowledge, techniques, decisions)
PROFES 2014, Helsinki, Finland 18
Page 10
12/12/2014
10
©Guenther Ruhe
AGENDAAGENDA
How it began: The Esprit Project PROFES How it began: The Esprit Project PROFES
Product Innovation Product Innovation
Analytical Open InnovationAnalytical Open Innovation
Analytics Case StudiesAnalytics Case Studies
The Road AheadThe Road Ahead
PROFES 2014, Helsinki, Finland 19
©Guenther Ruhe
New Products – Data & Information Needs
Profes 2014, Helsinki ‐© Guenther Ruhe
20
Information needs
Type of release planning problem
Features
Feature dep
enden
cies
Feature value
Stakeh
older
Stakeh
older opinion and
priorities
Release readiness
Market tren
ds
Resource consumptions
and constraints
What to release × × × × × × ×
Theme based × × × × × × ×
When to release × × × × × × ×
Consideration of quality requirements × × × × × × ×
Operational release planning × × ×
Consideration of technical debt × × × ×
Multiple products × × × × × × ×
Page 11
12/12/2014
11
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 21
Patternrecognition
Rough setanalysis
Cluster analysis
Morphologicalanalysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
PROFES 2014, Helsinki, Finland 22
Page 12
12/12/2014
12
©Guenther Ruhe
Value Synergies
PROFES 2014, Helsinki, Finland 23
In consideration of synergies
Without synergy considerations
Considering constraints is causing structural differences in plans and increase value (stakeholders feature points)
©Guenther RuhePROFES 2014, Helsinki, Finland 24
Re‐planning of not implemented features before starting Q2 with updated data from different customer groups
Re‐planning of not implemented features before starting Q3 with updated data from different customer groups
Re‐planning of not implemented features before starting Q4 with updated data from different customer groups
Time‐dependent Value
Page 13
12/12/2014
13
©Guenther RuhePROFES 2014, Helsinki, Finland 25
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 26
Patternrecognition
Rough setanalysis
Cluster analysis
Morphological analysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
Page 14
12/12/2014
14
©Guenther RuhePROFES 2014, Helsinki, Finland 27
1
2
ClustersClusters
Customization towards groups of customers
Having two cluster of customers
Having six clusters of customers
©Guenther Ruhe
Comparison of planning without clustering and by considering 6 clusters created from the crowd.
PROFES 2014, Helsinki, Finland 28
Page 15
12/12/2014
15
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 29
Patternrecognition
Rough setanalysis
Cluster analysis
Morphological analysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
©Guenther RuhePROFES 2014, Helsinki, Finland 30
Page 16
12/12/2014
16
PROFES 2014, Helsinki, Finland 31
ServiceID Service
S1 Live channel coverage
s2 Multiscreen
S3 Switch display
S4 Aspect ratio change
S5 EPG
S6 Remote control
S7 Support without touch screen
S8 Video on demand
S9 Youtube integration
S10 Source signal selection
S11 Variety of product usage model support
S12 Advertisement
S13 Archive
S14 Search
S15 Intuitive navigation
S16 Detect location
S17 Bookmarking
S18 Categorization
S19 Triple play
S20 Social network accessibility
S21 Playlist
S22 History
S23 Multicast
S24 Different views supportability
S25 Replay
S26 Instant streaming
S27 DRM
S28 Memory management
S29 Player integration
S30 Variety of quality support
S31 Parental control
S32 Channel preview
S33 Picture‐in‐picture
S34 Peer‐to‐peer wireless screen casting support
S35 Video recommendation
S36 Share content
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 32
Patternrecognition
Rough setanalysis
Cluster analysis
Morphological analysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
Page 17
12/12/2014
17
©Guenther Ruhe 33
Customer satisfied
Customer dissatisfied
Requirementfulfilled
Requirement not fulfilled
One‐Dimensional requirement
Attractive requirements
Must‐be requirements
(Berger et al., 1993)
Articulated specified measurable technicalNot expressed
Customer tailored Cause delight
ImpliedSelf‐evidentNot expressedObvious
©Guenther Ruhe
OTT Services ‐ Kano Questionnaire
PROFES 2014, Helsinki, Finland 34
How would you feel if “Support of Video‐on‐Demand (VOD)” was provided with this mobile app?
How would you feel if “Support of Video‐on‐Demand (VOD)” was NOT provided with this mobile app?
______ I like it that way ______ It must be that way ______ I'm indifferent ______ I can live with it that way ______ I dislike it that way
______ I like it that way ______ It must be that way ______ I'm indifferent ______ I can live with it that way ______ I dislike it that way
Functional formof the question
Dysfunctional formof the question
https://qtrial2014.az1.qualtrics.com/SE/?SID=SV_eeMrc9WjpFX6ZKd
Page 18
12/12/2014
18
©Guenther Ruhe
Kano Evaluation Table
PROFES 2014, Helsinki, Finland 35
Customer Requirements
Dysfunctional questions
Like Must‐be NeutralLive with
Dislike
Functional questions
Like Q A A A O
Must‐be R I I I M
Neutral R I I I M
Live with R I I I M
Dislike R R R R Q
Must‐be (M) One‐Dimensional (O) Attractive (A) Indifferent (I)Reverse (R) Questionable (Q)
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 36
Patternrecognition
Rough setanalysis
Cluster analysis
Morphological analysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
Page 19
12/12/2014
19
©Guenther RuhePROFES 2014, Helsinki, Finland 37
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 38
Patternrecognition
Rough setanalysis
Cluster analysis
Morphological analysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
Page 20
12/12/2014
20
©Guenther Ruhe
New Product (Super App) Design
PROFES 2014, Helsinki, Finland 39
M O A R I
2 2 1 2 0
{S4,S10,S11,S14,S20,S26,S27}
Value Effort
1570 261
M O A R I
1 1 3 1 3
{S1,S2,S3,S4,S5,S6,S7,S14,S19,S21,S22,S23,S25,S28,S32}
Value Effort
4506 261
©Guenther Ruhe
Release Readiness Optimization
12/12/201440
0.55
0.6
0.65
0.7
0.75
0.8
142 149 156 163 170
Calculated readiness
Projected readiness onrelease dateR
eadi
ness
Development time (days)
0.00 0.20 0.40 0.60 0.80 1.00
Number of feature implemented
Percentage of successful builds/integrationCode Churn per contributor per day
Defect find rate for last two weeks
Percentage of defect fixedDefects/KLOC
Test coverage: Covered LOC/ LOC
Number of code smells per class
Percentage of duplicated codeAverage method complexity
Percentage of issues fixed ()
Level of attribute satisfaction
Page 21
12/12/2014
21
©Guenther Ruhe
Release Readiness Optimization (2/2)
12/12/201441
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 42
Patternrecognition
Rough setanalysis
Cluster analysis
Morphological analysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
Page 22
12/12/2014
22
©Guenther RuhePROFES 2014, Helsinki, Finland 43
(X*Y)*X*
(YXm)n
XnYmYm(YX)nXl
(XYm)n
(XYX)n
(YX)nX(XY)n
(XmY)nXmYnXl
YnXm
XYnXm
YXn YnX
(YX)nXm
(XY)nXm
(XXY)nXm
154
4
49
358
3814
59
21
25
132
814
3
21
284
41
15
(YnX)m9
379
971
839
1809
76
56
77 154
2779
738
©Guenther Ruhe
AGENDAAGENDA
How it began: The Esprit Project PROFES How it began: The Esprit Project PROFES
Product Innovation Product Innovation
Analytical Open InnovationAnalytical Open Innovation
Analytics Case StudiesAnalytics Case Studies
The Road AheadThe Road Ahead
PROFES 2014, Helsinki, Finland 44
Page 23
12/12/2014
23
©Guenther Ruhe
Text mining
PROFES 2014, Helsinki, Finland 45
Patternrecognition
Rough setanalysis
Cluster analysis
Morphological analysis
Simulation
Optimization
CrowdsouringAnalytical Kano model
©Guenther RuhePROFES 2014, Helsinki, Finland 46
Page 24
12/12/2014
24
©Guenther RuhePROFES 2014, Helsinki, Finland 47
PPDINNOVATIVE PRODUCTS PROCESSES
Innovative products through innovative processes
PROFES 2014, Helsinki, Finland 48
Innovative Products through AOI
INNOVATIVE PRODUCTS
PROCESSES
• Acquiring innovationfrom external sources
• Analyzing data
• Integrate innovation
• Commercializing innovations
Page 25
12/12/2014
25
©Guenther RuhePROFES 2014, Helsinki, Finland 49
We strive to dothe best we can with the evidence at hand, but we accept that evidence may be incomplete, noisy, and even wrong
If you have certain patterns in mind, you will look for supporting evidence naturally.
So ask for anti‐patterns!Data science should
be about causation, not correlation (watch for bias and confounding
factors!)
Data, analyses, methods and results have to be publicly
shared
Don't show me what is; show me what to do
Your project has a history. Learn from it.
Decide from it. Embrace it!
We will be able to gain insights from
the past to improve the
future
Good data science does not get in theway of developing
software but supports it (makes it more efficient)
Underlying theory needs to inform the
data analysis
SE data sciencesshould be actionable,
reproducible.
SE data sciencesshould be actionable,
reproducible.
What Counts is Insight not … Numbers
©Guenther Ruhe
References[1] Nayebi, M and Ruhe, G (2015), “Analytical Product Release Planning”, accepted to be
published in the book “The Art and Science of Analyzing Software Data: Analysis Patterns”, C. Bird, T. Menzies, and T. Zimmermann (eds.), Kaufman & Morgan 2015.
[2] Nayebi, M and Ruhe, G (2015), “Analytic Open Innovation for Trade‐off Service Portfolio Planning – A Case Study on Mining the Android App Market”. Submitted to Special Issue on Software Business, JSS
[3] Nayebi, M. (2014), “Mining Release Cycles in the Android App Store”, The 36th CREST Open Workshop on App Store Analysis, London, England
[4] S. Alam, S. M. Shahnewaz, D. Pfahl, and G. Ruhe, “Analysis and Improvement of Release Readiness ‐ A Genetic Optimization Approach,” Proceedings of Product Focused Software Development and Process Improvement (PROFES), 2014
[5] Workshop on Data Analytics, Dagstuhl 2014
[6] Chesbrough, H., “Open Innovation: The New Imperative for Creating and Profiting from Technology”, Harvard Business Press, 2003.
[7] Ritchey, T. , "Wicked Problems‐‐Social Messes: Decision Support Modelling with Morphological Analysis.," Springer 2011.
Profes 2014, Helsinki ‐© Guenther Ruhe
50