The Analysis Part of Integration Projects
Post on 28-Jul-2015
190 Views
Preview:
Transcript
Connected Systems Consulting Ltd
Sponsored & Brought to you by
The Analysis Part of Integration ProjectsMichael Stephenson
https://twitter.com/michael_stephen
https://www.linkedin.com/in/michaelstephensonuk1
The Analysis Part of Integration Projects
Connected Systems Consulting Ltd
Connected Systems Consulting Ltd
Who am I?Michael Stephenson
– UK-Based Freelance Consultant specializing in:• BizTalk• Windows Azure• Integration
– Microsoft Integration MVP for 6+ years– Pluralsight Author– Azure Insider/Advisor– One of organizers of UK Connected Systems User Group– Founder of BizTalk Maturity Assessment – www.biztalkmaturity.com– Worked on approx. 30 projects that have leveraged Azure– First project went live about 4 years ago
– Contact Info:• Blog: http://microsoftintegration.guru• Twitter: @michael_Stephen• Linked In: http://www.linkedin.com/in/michaelstephensonuk1• Email: michael_stephensonuk@yahoo.co.uk
Thanks to….
Question
“How often do you feel thrown in at the deep end when starting an integrationdevelopment project because no one seems to have a clear understanding ofwhat is required”?
+1 in the chat window if you think this is common
The Usual Story
Common Scenario 1
“I need a feed of customer data from system A to system B”
Err…. Ok so what does this mean?
Common Scenario 2
Contoso
Fabrikam
Acme
CRM System
ERPSystem
FinanceSystem
MarketingSystem
ProductsSystem
ProductsSystem
OrdersSystem
OrdersSystem
OrdersSystem
CustomersSystem
Its this interface here, can we just do this?
Those arrows are pretty but we don’t know anything about what it means
Common Scenario 3
“I need an additional data attribute added to an Existing integration process”
This may be straightforward and we could just get on
with it… but lets get a little info to be sure
Common Scenario 4
“We need to change the existing new customer processSo it also updates the new CRM system”
I hope we have all of the data we
might need
The Problem
What systems are involved?
What does the data look like?
How much data do we
need to process?
What do things happen?
What is the process flow?
Are there any alternative or
exception scenarios?
How do I talk to each
application?
Are there any SLA’s?
Is there any business logic?
What are the data
transformation rules?
Will we need any new
infrastructure?
Are there any performance
requirements?
Are there any security
requirements?
Is there any documentation
?
When do things
happen?
“There’s just so much unknown that affects how we design and build the system”
How Important is Analysis?
Integration is easy when…• The solution is very like something you have done before
• You are reusing stuff you have build in the past
• The applications involved are well understood
• Only a small number of people are involved
• The data is not complicated
• The process logic is simple
• You have a good test environment setup
• Everyone understands the process to be built
• You have good testing
• Your dependencies are well understood
Integration is hard when…• You need new infrastructure
• You have lots of dependencies or they are poorly understood
• Lots of external vendors or people are involved
• You need to use patterns you have not implemented before
• The applications involved are not well understood
• The applications involved are unreliable
• You have a poor test environment setup
• You follow good ALM practices
• You don’t have enough information
• Your requirements are not well understood or clear
• You have a poor or incorrect solution design
• Your making it up as you go along
Getting it wrong
Not enough analysis• Developer gets things wrong
• Architect makes bad assumptions
• Poor documentation
• Testing problems
• Delays to redo and fix things
• Developer ends up chasing around to do analysis that wasn’t done
• Takes longer to deliver value
• Unhappy team
• Unhappy customer
Too much analysis• Wasted time
• Too much documentation
• Lots of the information doesn’t add value
• Takes longer to deliver
• Costs too much
• Over complicates things
• Difficult to deal with change
• Time to delivering value is high
• Unhappy customer
Amount of Analysis
Amount of Analysis
High
HighLowSweet spot = Just enough
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 300
20
40
60
80
100
120
Chart Title
Cost of Project Risk
We are usually in one of these places
MethodologiesI have a methodology doesn’t this solve the problem?
Methodologies – In Theory
Waterfall Agile
Analysis
DesignDevelop
Analysis Design Develop
Methodologies – In Practice
Waterfall AgileAnalysis & Developme
nt
Re-doing Analysis & Develop that
we missed or got wrong
Analysis Design Develop
Analysis Design Develop
Time Spent
Value Delivered
What happened to architecture?
Agile v’s Waterfall
Agile• Just give it to the scrum team
• We are doing agile so things are assumed to be simpler
• The team will figure it out
• Often teams don’t do analysis sprints
• Often This should be a sprint 0 type activity
• You usually have more stakeholders than just the product owner
• Encourages POC of risk areas which is good
Waterfall• Lots of analysis and design before any real world
validation
• Analyst/Architect not close enough to delivery to handle change
• Analysis sometimes too detailed and overkill in terms of presentation format
• Analysis that doesn’t add value
“In the real world a methodology often doesn’t solve the problemas they aren’t getting the right amount of the right information”
Doing it effectively
Lets take the best things and make
them work?
“Gathering the information we need to allow us to design, build and test the solution the customer needs, and being able to communicate that information to the entire team”
Analysis is….
“Just enough analysis to do an effective job”
“Analyst and Architect working together”
The Project
Developer
Legacy ApplicationTeam
ERP System Team
Infrastructure Team Support Team Security Team
Finance Team
Project ManagerSenior Management
Change Team
Definition of Ready
Do we feel ready?• Do we feel confident about what
we need to do• Is the project broken down into
stories or features?• Do we need or have the right
analysis artefacts
Measuring Readiness• INVEST• I – Independent (of others)• N – Negotiable• V – Valuable• E – Estimatable• S – Small• T - Testable
Analysis Iterations
• Iteration 1 – High Level• Iteration 2 - More Detail• Iteration 3 – Reduce Risk
Iteration 1
Iteration 2
Iteration n
Iteration 1 - Analysis
Step 1 – The User Story
StructureIn order to (why is it important)As a ………. (who wants it)I want …… (what do you need)
ExampleIn order to collect money from customers
As a sales director
I want the CRM system to process credit card payments on a monthly basis
“If you cant explain the value statement then why do we want to do it”?
Step 2 – The Context Diagram
Old CRM New CRM
Orders Website
Lookup Customer
Customer Transfer
Orders System
Process Orders
Step 3 – Integration Use Cases
Step 4 – High Level Process Logic
• Could you easily explain the process logic to someone
Step 5 – High Level Data Items
• What entities?
• Any relationships?
• Any key data attributes?
• Where does data come from?
Step 6 – Applications Involved
• What are the supported interfaces?
• What protocols are supported?
• Where are the applications?
• Who manages them?
• Who are the key contacts?
Step 7 – Non-Functional Requirements
• Is there a general list of NFR’s
• Are there any specific NFR’s raised by the business
• Are their any IT NFR’s
• How should the system perform?
Step 8 – Stakeholder Map
Keep Satisfied
• CRM Development Team
• Infrastructure Team
Actively Engage
• Product Owner• HR Department• Support Team
Monitor
• Warehouse Dept
Keep Informed
• ERP Development Team
• Finance Dept
• Make sure we identify stakeholders and where they fit into the map so we can make sure we talk to the right people at the right time
• Make sure owners of error conditions are identified not just owners of processes
Step 9 – Which type of Integration?
• API & Services
• ESB / Messaging
• Batch Based
• Simple Orchestration
• Complex Orchestration
• Mixed
Step 10 – Candidate Architecture
• Is it something we already do?
• What patterns do we need?
• How might we build it?
• How confident are we about the design?
Step 11 – Create Integration Catalogue• Agree a friendly Interface Name
• Identify Source and Destination Systems
• Identify Candidate Integration Type• Batch• API• ESB/Messaging• Complex Orchestration
• Identify Types of Data involved
• Identify grouping of interfaces
• Identify related Use Cases
• Relate non functional requirements
• Relate candidate architecture
Checklist
What have we done• Created an Integration Catalogue for project which
pulls together:• User Stories• High Level Context• Identified Use Cases• Draw Process Logic• High Level Data Model• Applications Involved• Stakeholders• Non-Functional Req’s• Candidate architecture
How do we feel• Are we confident about process
• Are we confident about data
• Are we confident about candidate architecture
• Does it feel simple or complex
• Do we feel we have a good understanding
Iteration Summary
Iteration 2 – Analysis & Design
“We have decided we need more information & detail before proceeding, what do we get next?”
What next?
Where do we look – Key Areas
1 Processing Logic • What processing logic is required
• What data transformation is required
• Are there any patterns
2 Application Connectivity
• How might we connect to the application
• Is the application interface well defined and easy to work with
3 The Data • Does the data make sense• Does the data map between
systems well
1
2
2
2
2
3
3
3
Step 1 – Flush out features
• Find alternative scenarios
• Specification by example
• Find exceptions
• Confirm owner of exception scenarios
Step 2 – Define Data
• Get sample data
• Data Model
• Get data specifications if existing• XSD/XML• WSDL/Swagger• JSON• Flat File Structure• HL7• EDI
Step 3 – Define Data Mappings
• Encoding formats
• Reference data mappings (Look up values)• Mr AA• Mrs AB• Miss AC
• Number formatting• Leading zeros• Decimal places
• Text• Length• Alignment• Padding
• Date formatting
Important this can often be an areaWhich can throw up more integration use cases or maintenance challenges
Step 4 – Application Integration Capabilities
• Do we have connectors?
• Are the connectors well understood?
• Are there challenges doing the connection?
• How will security work?
• Are there any throttling requirements?
Step 5 – Non Functional’s
• Revisit the NFR’s and validate with stakeholders
Step 6 – Patterns
• Are there any obvious integration patterns involved
• Patterns can help create a common understanding
Step 7 – Design Decisions
• Are there any key design decisions to make• Who will be involved• How will the decision be made• How do we communicate to the decision to everyone• How do we ensure the decision is followed?
Step 8 – Update Candidate Architecture
• Is our candidate still good?
• Are there any new things
Checklist
What have we done• More detailed process definition using
Specification by Example
• Defined data in more detail
• Define mappings
• Reviewed NFR’s
• Identify Patterns
• Update Architecture
How do we feel• Are we confident about process
• Are we confident about data
• Are we confident about candidate architecture
• Do we understand the complexity
Iteration Summary
Iteration 3 – Reduce Risk
“How do we reduce the risk around the areas we have concerns with”
What next?
Step 1 – Logical Process POC
• What to do• Implement light weight stubs to prove the applications will
work
• What does it achieve• Allows focus on “will the applications work”• Mitigates risk that the process logic will not work
Step 2 – Application Integration POC
• What to do• Test we can connect to the application• Create BDD style Specflow tests of the Application interface
• What does this achieve• Test the application behaves the way we expect it to• Tests the data looks like what we expect it to• Tests assumptions about error conditions
Step 3 – Update Candidate Architecture
• Is our candidate still good?
• Are there any new things
Checklist
What have we done• POC the key risk areas
• Updated architecture candidate
How do we feel• Do we have any concerns?
• Have we mitigated risks?
Iteration Summary
Analysis Iterations
• Just enough analysis to set us up to be successful
• Use the iterations that are needed
• Make sure information is clear and can be understood by anyone in the team
Iteration 1
Iteration 2
Iteration n
Takeaway
• Define your organisations definition of ready?• What analysis artefacts do you usually need?• Share it?• Who is up for some community guidance around this?• Technet wiki or something?
top related