1 Lean Integration: Translating an Innovative Agile Approach into Business Value MIS Research Center Seminar March 25 , 2011 John Schmidt VP, Global Integration Services 1 Informatica The #1 independent leader in Data Integration • 2009 Revenue: $500 million • 5-year Average G hR 18% $500 $550 Growth Rate: 18% per year • Customers: 3,931 • 84 of Fortune 100 • 87%+ of Dow Jones • Government organizations in 20 countries • Partners: 400+ $250 $300 $350 $400 $450 2 • Major SI, ISV, OEM and On-Demand Leaders • Employees: 1,755 • Direct in 28 Countries $150 $200 $250 2005 2006 2007 2008 2009
27
Embed
Lean Integration: Translating an Innovative Agile … Integration at MISRC[1...1 Lean Integration: Translating an Innovative Agile Approach into Business Value MIS Research Center
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Lean Integration:Translating an Innovative Agile Approach into Business Value
MIS Research Center SeminarMarch 25 , 2011
John SchmidtVP, Global Integration Services
1
InformaticaThe #1 independent leader in Data Integration
• 2009 Revenue: $500 million
• 5-year AverageG h R 18%
$500
$550
Growth Rate: 18% per year
• Customers: 3,931
• 84 of Fortune 100
• 87%+ of Dow Jones
• Government organizations in 20 countries
• Partners: 400+$250
$300
$350
$400
$450
2
• Major SI, ISV, OEM and On-Demand Leaders
• Employees: 1,755
• Direct in 28 Countries $150
$200
$250
2005 2006 2007 2008 2009
2
Our Singular MissionEnabling The Information Economy
We enable organizations to reduce IT costs and gain a competitive advantage in today’s global information economy by empowering them to access, integrate and trust all their
information assets.
3
The Role of IT in Business
“CEO’s CFO’s and CIO’s all recognize there isCEO s, CFO s and CIO s all recognize there is zero separation between business strategy and IT execution today. There is pretty much nothing you can do, from cutting costs or growth without the enablement of IT in one way, shape or form.”
S S h k b k
4
Steve Schuckenbrock, President, Dell Large Enterprise Business
Reference: Financial Times Special Report, Wednesday, October 27, 2010
3
The Data Driven Enterprise
Partner Trading Network(B2B)
Cloud Computing TraditionalEnterprise
HIPAA
SEPA
NACHA
5
Problem: Data chaos results from unnecessary complexity
Root Cause - Integrations manufactured as custom point solutions
6
4
Craftsman Approach – Custom Development
7
Manufacturing WorldSkilled manual workersManufacture by hand
Custom products (works of art)
Integration WorldSkilled IT Specialists
Hand codingCustom integration points
Assembly Line Approach – This is ICCs now
BenefitsConsistent Quality
Lower Cost
8
Integration WorldDedicated skilled labor
Standard ToolsPeriodic Large Releases
Manufacturing WorldDedicated skilled labor
Standard ToolsMass production
5
Modern Factory Approach – Automated Flow of Materials and Information
BenefitsLow cost AND high qualityExactly the way you want it
It is Critical That You Have an Expert and Sustainable Integration Capability
Efficiency Spectrum
Hand codingIntegration
Tools (EAI,...)
Projects& Ad hoc
integration
ProgramManagement
Operational Efficiency
IntegrationTechnology
ManagementPractices
Low High
Integration Platform
Competency Center
Integration Factory Platform
LeanIntegration
10
Fast(quick & dirty)
ImprovedQuality
CostEfficiency
Faster, Better AND Cheaper
BusinessBenefits
Point-in-time Methods Sustaining Methods
1990’s 2000’s 2010’s
6
Cultural Change is easy – isn’t it?
11
Transform complexity and chaos to an Agile, Data-driven Enterprise
Competitive Weapon
Sharedf Dat
a
No Compromise
Complexity Agility
Shared Resource
Controlled cost
Unmanaged silos
Perc
eptio
n of
Business Empowerment
ANDIT Control
StandardizationAND
Innovation
12
Methodology
Transformations
Technology Hand coding IntegrationTools
Integration Platform
Integration Factory Platform
Independent toInterdependent
Tactical toStrategic
Managed toCultural
Project Program CompetencyCenter
LeanIntegration
7
Analyst Perspective
“Taking a production line approach to integrations makes aTaking a production line approach to integrations makes a lot of sense…they should be more widely considered and implemented.”
Philip Howard, Research Director - Data ManagementBloor Research, January 7, 2009
13
Lean Integration Principles
Eliminate Waste Continuous Improvement
Automate Processes Empower the Team
14
Build Quality In
Optimize the Whole
Plan for Change
8
1. Eliminate WasteExample: Value Added Ratio – Before Lean
Scenario: Simple Change Request to the Data Warehouse
5 min 5 min 2 hours 1 hour 1 hour 50 minValue-Add
15
1 week 2 weeks 2 weeks 1 week 2 weeks
NonValue-Add
1. Eliminate WasteExample: Most non-value added activities eliminated
5 min 5 min 2 hours 1 hour 1 hour 50 min
4 hours 8 hours 0 weeks 4 hours 16 hours
Value-Add
NonValue-Add
16
Lead Time Reduction = = = 97% Original Lead Time – Current Lead Time
Original Lead Time
1344 - 32
1344
Lead Time Reduced from 2 months to <1.5 days
9
Archiving reduces datafile size (GB)
1,000
1,200
1,400
Without archiving With archiving …and truncating temp tables
1. Eliminate WasteKeeping data in production that no-one uses is waste
0
200
400
600
800
Current Year 1 Year 2 Year 3
Data that is not used still costs a lot:• Disk capacity (production, test, DW)
B k t (t l b t )
Projected savings by eliminating data waste
$12,907,437
17
• Backup costs (tapes, labor, storage)• Business slow-down by end-of-period jobs• Application upgrade costs• Impact on real-time transactions• Compliance costs• Data quality analysis and resolution time
$1,603,277
$4,925,769$6,378,391
Year 1 Year 2* Year 3 3-year Total
Take Away: Eliminating Waste is not about cutting out fat
“D ’t thi k f l i i di t t l i ht thi k“Don’t think of lean as requiring a diet to lose weight; think of it as focusing the people, process, technology, and architecture that you possess on the projects that have the most impact for the business.”
Mike Gualtieri, John Rymer, Jeffrey Hammond, Top Five Changes For Application Development
18
Forrester, January 4, 2010
10
2. Automate ProcessCreate assembly lines for rapid customization
• Generating custom wizards for templates to maximize productivity for end users– developers and/or analysts
• Ability to build your own wizards based on patterns without coding
IT Designs Templates and Patterns – Users create custom solutions fast
y y p g
• Common data integration and event processing patterns
2. Design Template1. Understand and Define Common Patterns
3. End Users Quickly Configure and Customize in Assembly Line-Fashion
• Data file interchange with supply chain partners
• Time-based event correlation
Take Away: Automation is a key to simplification
“Factory IT couples lean management techniques y p g qand process improvements with advances in cloud computing and software development to simplify the operating environment and improve productivity and cost performance.”
Reshaping IT management for turbulent times,Roger Roberts,Hugo Sarrazin, Johnson Sikes, McKinsey
20
Quarterly, November, 2010
11
3. Empower the TeamProblem: IT is the bottleneck for data requestsSolution: Give the business self-service capabilities
Business Self-Service
Enables business self-service, improving
business agility
21
business agility
IT maintains governance and oversight
IT personnelSupport
Take Away: Self-service improves business agility AND gives IT better governance
“Lean is about ensuring IT is more closely aligned to the Business.”
Stephen Pritchard, Financial Times
22
Source: Financial Times Special Report, Wednesday, October 27, 2010
12
4. Continuously ImproveProblem: How can groups improve themselves?Solution: Use visual management of the entire integration lifecycle.
Object reuse Project X Development Project Y Development
Data Quality MetricsData Volumes by ProjectDevelopment Activity
23
Take Away: Don’t use Lean to cut staff
“Best-practice executives view lean as a performance improvement strategy rather than merely a cost-cutting exercise.”
Alexander Peters Forrester Research
24
Alexander Peters, Forrester Research
Source: Financial Times Special Report, Wednesday, October 27, 2010
13
5. Build Quality InProblem: Quality issues accumulate without business involvementSolution: Involve business in fixing highest priority issues
Line of business manager
Data Quality
Data Steward
25
ITDeveloper
Take Away: Metadata is an enabler for IT automation
“Metadata allow systems to adapt more quickly toMetadata allow systems to adapt more quickly to changing requirements by pushing power into the data, and out onto users.”
Brian Foote and Joseph Yoder, Big Ball of Mud, Addison-Wesley Software Patterns Series, 2000
26
Reference: Financial Times Special Report, Wednesday, October 27, 2010
14
6. Plan for ChangeInsulate consuming applications from change
Applications PortalsAnalytics
Service ServiceService
Service
ServiceServiceServiceService
Service
ServiceService Service
ServiceService ServiceService
ServiceService Service
ServiceService
CUSTOMER ORDER PRODUCT INVOICE
27
Build Quality
In
ContinuouslyImprove
Optimize The
Whole
PlanFor
Change
Focus on Customer Value and Eliminate
Waste
Lean IntegrationProven Value Proposition: Better, Cheaper & Faster
AutomateProcesses
Empower the Team
Cloud Services The InformaticaPlatform
Mapping ArchitectFor Visio
PCAE - Metadata Manager
Data QualityV9 Data ServicesDatabase Archiving
28
Implemented data quality metrics and monitoring system;
Saved est.$3M in development costs
and avoided $20M in AML regulatory fines
PlatformFor Visio Manager
Saved $1.4M in first 6 months because of improved customer master data quality; Reduced SKU’s by
40% by reducing old or obsolete parts
Centrally managed virtual views
speed up time to add products to
DW portfolio from 1700 hours to 40
hours
Business analysts use Informatica
Cloud Services to self-serve data for their own analysis, cutting lead-times to information by
90%
4 month payback from storage reduction,
and rapid retrieval of archived claims for improved customer
service
After re-working their ETL processes to always use design patterns, achieved 8x efficiencies on
all new projects and re-work on old
projects
Reduced impact analysis time
between 85-95%, reduced time for business analyst inquiries by 50%, reduced end-user support time by
over 65%
Informatica Confidential and Proprietary
15
Manufacturing Production Lines
Bread factoryBottling Line
Car Assembly Line
29
Leading Practice: Integration Systems are Managed Like First-Class Business Systems
Business SystemsIntegration Systems
IntegrationHub
TransformationHub
Application Integration
Enterprise Resource Planning Systems
Sales & Marketing Systems
CustomerSystems
Product and Order Management
Systems
Integration Systems
Human Resource Systems
Finance & Accounting
Systems
Distribution and Inventory Systems
External Systems (Software as a
Service)
Engineering & Manufacturing
Systems
Integration Repository
Portal Process Management
DataWarehouse
Business Analytics
Master Data Management
Extract Transform Load
Data Services
Managed File Transfer
Enterprise Service Bus
Information Security
Enterprise Messaging
InformationPortal
&
Metadata Manager
Process State Engine
CloudServices
Business to BusinessData Quality
30
IntegrationHub
TransformationHub
Integration RepositoryPortal
Application Integration
Process Management
Directory &Single Sign-on
Content Management
Business Glossary
Service Registry
ActivityMonitoring
Workflow
Search Complex Event Processing
Data Stewardship
16
Informatica Platform Mapping
Integration SystemsIntegration
HubTransformation
HubApplication Integration
DataWarehouse
Extract Transform Load
CloudServices
UltraMessaging
CEP
Integration Repository
Portal Process Management
Business Analytics
Master Data Management
Data Services
Managed File Transfer
Enterprise Service Bus
Information Security
Enterprise Messaging
InformationPortal
Metadata Manager
Process State Engine
Business to BusinessData Quality
PowerCenter
B2B DataExchange
ILM
CloudServices
B2B DataExchangeB2B DataExchangeB2B DataExchange
PowerCenterPowerCenterPowerCenterPowerCenter
31
Directory &Single Sign-on
Content Management
Business Glossary
Service Registry
ActivityMonitoring
Workflow
Search Complex Event ProcessingData Stewardship
PowerCenter
Data Quality
MDM
PowerCenterPowerCenterPowerCenterPowerCenter
Data Quality
Informatica Factory Platform Products in Eight Growth Categories
Data Quality Informatica Cloud PlatformData Data Data Data Quality
Global Address Informatica Cloud PlatformAssessmentLoader SynchronizationReplication
Global Address Verification
Informatica Cloud Platform
33
Data Explorer
Data QualityData Transformation
Data Exchange
IdentityResolution
Data Services
PowerCenter
Data Archive
Data Subset
Data Privacy
RulePoint
RuleCast
Real-TimeAlert Manager
PowerExchange MDMAdvanced XMLLBM Streaming
UME Persistence
UME Queueing
Production Line for Supply Chain Integration
Data Quality Data Data Data Data Quality
Global Address Informatica Cloud PlatformAssessmentLoader SynchronizationReplication
Global Address Verification
Informatica Cloud Platform
34
Data Explorer
Data QualityData Transformation
Data Exchange
IdentityResolution
Data Services
PowerCenter
Data Archive
Data Subset
Data Privacy
RulePoint
RuleCast
Real-TimeAlert Manager
PowerExchange MDMAdvanced XMLLBM Streaming
UME Persistence
UME Queueing
18
Production Line for Channel Integration (ESB)
Data Quality Data Data Data Data Quality
Global Address Informatica Cloud PlatformAssessmentLoader SynchronizationReplication
Global Address Verification
Informatica Cloud Platform
35
Data Explorer
Data QualityData Transformation
Data Exchange
IdentityResolution
Data Services
PowerCenter
Data Archive
Data Subset
Data Privacy
RulePoint
RuleCast
Real-TimeAlert Manager
PowerExchange MDMAdvanced XMLLBM Streaming
UME Persistence
UME Queueing
Production Line for Business Intelligence COE
Data Data Data Informatica Cloud PlatformData Quality Data Quality
Global Address Loader SynchronizationReplication
Informatica Cloud PlatformAssessment
Global Address Verification
36
Data Explorer
Data QualityData Transformation
Data Exchange
IdentityResolution
Data Services
PowerCenter
Data Archive
Data Subset
Data Privacy
RulePoint
RuleCast
Real-TimeAlert Manager
PowerExchange MDMAdvanced XMLLBM Streaming
UME Persistence
UME Queueing
19
Good technology alone does not guarantee success
“Deploying technology, by itself, is a poor strategy for improving the implementation of interfaces. An effective strategy for integration is established by using these critical success factors: • Establish an ICC. • Develop an integration "city plan" consisting of:
• Policies that implement the necessary governance • A set of best practices that is continually refined by measuring
the result of applying the best practices in integration projects • A technology architecture that establishes standards for the
technologies to be used during application integration
37
• Have integration competency staff drive the selection of the integration technologies that are to be deployed. “
The Seven Critical Success Factors for Application Integration,Jess Thompson, Gartner, November 10, 2010
Lean methods in a nutshell
38
20
Lean methods – a more complete list
39
As-Is Value Stream Map
CR Review Committee
GMNA Applications Team(Irina)
Data Warehouse Team
Change Request
Confirmation Request
Telephone Tag
Status Request
Status Update
Automated Workflow/Tracking (Cust satisfaction)
Data Dictionary to clarify rqmnts N tif C t
Status Request
Status Update
DOIT Corporation: Value Stream Map (AS-IS) for Change Request ProcessMonday, December 20, 2010
Semi-Weekly Review
Requirements Review
Production Deployment
Test ExecutionTesting HandoffDesign & Development
Integration Team Manager
Assign Resource
Test Team Manager
Test SchedulingData Warehouse Team
Production CR Submission
Production Execution
Add CRTo List
ApprovedChanges
ForwardCR Request
To Developer
Clarify Requirements
Requirements Clarification
Approved CR& Design Docs
Design Docs& Schedule
Change Management Board
CR Approval
Daily ETL Batch Run
Test Case Development
Status Request
Status Update
Test Results Distribution
CR Approval& Schedule
Charge Request
Bypass Committee for
simple changes (8 days)
to clarify rqmnts (13 days)
Design Document
Approved Designs
Bypass Council for simple CR’s
(26 days)
Notify Customer(Cust Satisfaction)
(5 days)
Automated Regression
Testing(21 days - requires
investment)
Architecture Review Council
Design Approval
40
Development Team
1
Review
Infrastructure Team
DeploymentTest TeamDevelopment Team
Development Team
Development
CR’sP1x12P2x35P3x124
Automatic Daily ETL Batch Run
Execution
30 Minutes
8.8 Days
15 Minutes
8.5 Days
90 Minutes
12.8 Days
180 Minutes
1 Day
15 Minutes
26 Days
180 Minutes
13.3 Days
Work Time = 510 Minutes (8.5 hrs or 0.35 Days)
Lead Time = 75.6 Days0.3 Days
Test Team
Development
Value Ratio: Work Time / Lead Time = 0.5%Notes:(1) Lead Time includes 5 delay in customer notification(2) Lead Time could be reduced to 24 days with just process changes and using existing tools(3) Lead Time could be reduced to 3 days with a capital investment for automated testing
21
Integration Factory Organizational Structure
41
Helpful resources: Best practice referencesAvailable at amazon.com or integrationfactory.com
• Integration Competency Center (2005): A permanent cross-functional team operating as a shared service function supportingas a shared service function supporting multiple organizational units and sustaining integration in a coordinated manner.
• Lean Integration (2010): A teachable, sustainable management system that
42
g yemphasizes creating value for customers, continuous improvement, and eliminating waste as a sustainable data management and integration practice.
22
43
CIO Magazine, October 2010Staff Pick for IT Management & Leadership
44
23
Sharing Knowledge Among Users
The go-to destination to buy and sell proven data integration, data quality, and data data integration, data quality, and data management solutions (Blocks).
Get promoted! Reduce your time to market with pre-built solutions
Get ahead of the competition! Increase your sales reach with Informatica Marketplace
45
Maximize your Informatica investment with Informatica Marketplace - www.informaticamarketplace.com
Integration Maturity Assessment Survey
Integration Maturity Assessment
50.0
60.0
e LeanLean
Integration Maturity Assessment
50.0
60.0
e
0.0
10.0
20.0
30.0
40.0
Financia
l Man
agem
ent
Integ
ration M
ethod
ology
MetaData
Manage
ment
Inform
ation
Archite
cture
Busines
s Proce
ss
Modelin
g Mana
gement
Integ
ration Sys
tems
A f S b 1 2010
MIN
- M
AX -
Aver
age Lean
Sustaining
Program
Project
Lean
Sustaining
Program
Project0.0
10.0
20.0
30.0
40.0
Financia
l Man
agem
ent
Integ
ration M
ethod
ology
MetaData
Manage
ment
Inform
ation
Archite
cture
Busines
s Proce
ss
Modelin
g Mana
gement
Integ
ration Sys
tems
MIN
- M
AX -
Aver
age
46
http://vip.informatica.com/?elqPURLPage=7960
As of September 1, 2010
24
Impact: IT Costs are Increasing
47
Takeaways
• Lean is Easy – don’t try to be “perfect”
• Learn a Lean technique and start using it• Learn a Lean technique and start using it
• Invest in Factory tools
48
25
49
Reference Models are an essential element of an effective Business Framework
Solutions defined in terms of
How does a framework help?
• Faster projects
• Higher quality solutions
IFW FOUNDATION MODELS
Solutions defined in terms of
Solutions defined in terms of
• Decreased cost through reuse of technology assets
• Tighter integration of business and IT – more
flexible and agile organization
• Common Business Language
• Up-to-date Risk and
50
IBM’s IFW
FSDM FSFM FSWM
IFW PROCESS & INTEGRATION MODELS
IFW PROCESS MODELS
FS-BOM
FS-IDM
IFW DATA MODELS
BDWM
BSTs
IFW ODS
Banking
Solutions defined in terms of
Up to date Risk and Compliance requirements
integrated throughout
26
Case Study: Wachovia’s Business Architecture
51
Information Architecture Framework
Layer 4 – Enterprise ViewOverarching consistent context for executives & information stewards 4
repository of relevant models)Physical Data Models
File/Table PropertiesStorage Information
Logical Data ModelsEntity Relationship Diagram
Information ModelsBusiness GlossaryCanonical Model
Transformation ModelsInterface Specifications
Transformation rulesProtocol Definitions
Interaction ModelsLogical Data Map
Sequence diagrams
Process ModelsBusiness Event Model
Operational Workflow Model
Inte
grat
ion
View
s
Layer 3 - Business ViewDomain models for business owners and project sponsors
Layer 2 – Solution ViewArchitecture models for specific
systems and solutions
1R 1M
2R 2M
3R 3M
Layer 1 –Technology ViewTechnical models for developers,
engineers and operations staff
Bus
inMaster Data List4
52
Data at Rest Data in Motion
A layered approach is needed to manage data complexity, provide multiple stakeholder views, and maintain traceability to business requirements.
27
Data Governance Methodology
2
One-Time Effort Repeat for every program
10Implement Projects
10Implement Projects
1Organize
Governance Committee
Define Governance Framework
4Assign
Organizational
5Scope
Program
3Develop
Enterprise Models
6Assess
Baseline & Data Quality
8Plan Migration
Roadmap
10Implement Projects
7Develop Target
Architecture
9Develop Program Models
53
gRoles
Entry Points1. Enterprise Initiative (Large-scale program with defined ROI – e.g. Merger or major business transformation)2. Top-level Directive (C-level or higher – e.g. one view of customer/business, control IP, data security, etc.)3. Scale-up DQ and MDM Projects (Leverage successful solutions that demonstrate the value for a specific
business area and scale them up to cross-functional enterprise-wide initiatives)