events.techtarget.com Sam Strum, Director of Data Services, INTTRA Case Studies in Action Tips for Creating a Next- Generation Data Warehouse SearchBusinessAnalytics SUMMIT
events.techtarget.com
Sam Strum, Director of Data Services, INTTRA
Case Studies in Action
Tips for Creating a Next-
Generation Data Warehouse
SearchBusinessAnalytics SUMMIT
What Will Be Presented
Overview of INTTRA
INTTRA BI and DW Landscape as of early 2013
DW and BI Principles and Roadmap
How We Ended Up
Lessons Learned
Next Steps
SearchBusinessAnalytics SUMMIT | © TechTarget
INTTRA Company Profile
CARRIERS
& NVOCC 49
REGISTERED
USERS
220,000
CONTAINERS PER
WEEK
530,000
ALLIANCE
PARTNERS 109
COUNTRIES 130
SHIPPING
CONNECTIONS
1.65m
1 CONNECTION TO
WORLD’S LARGEST
SHIPPING NETWORK
SearchBusinessAnalytics SUMMIT | © TechTarget
INTTRA Value Proposition
BI Leadership Summit | © TechTarget
INTTRA Information Content Types
BI Leadership Summit | © TechTarget
Find a Voyage:
Search over 12.5M schedules from 30 of the world’s most
desired carriers through 1
connection.
Book Your Cargo:
Book directly from your search results
- no rekeying involved! We transfer your
selected schedule directly into your
booking – or if you know your vessel voyage, you can
enter booking information online.
Submit Shipping Instructions:
Shipping instructions are
automatically filled when you enter your booking
details. Our flexible design enables you
to start the shipment process
at either booking or shipping
instructions, and still automates the
rest of the shipment process.
Proof & Print Bill of Lading:
Get faster delivery of Bill of Ladings
(B/L) and minimize container shipment delays. Your B/L is always available whenever and
where-ever you need for fast and
easy access. Eliminate manual rekeying through the entire process and increase B/Ls
quality.
Track Shipment:
If you enjoy logging into multiple
systems to check the status of a
shipment for your customer, you
won’t like INTTRA. We give you a
single way to look up all your
container status events.
Manage Invoice:
Now that you’ve got a handle on
reducing your own paper, you can set
your sights on eliminating other
people’s paper too. Receiving invoices electronically saves time, money, and eliminates errors
that can be introduced with manual entry.
Now that you know so much can you think of
Analytical Use cases applicable to INTTRA?
BI Leadership Summit | © TechTarget
1. ?
2. ?
3. ?
4. ?
INTTRA Business Intelligence & Data
Management History
● Historically a transaction-focused organization
- Primary revenue driven from transaction/container volume
- An E-Commerce company
● Market share penetration has revealed an increasing opportunity to exploit
the value of the data captured in those transactions
- Cross Sell, Upsell, Targeted Marketing
- Carrier Ranking, Growth Opportunity for Carriers
● Market growth has presented new information stakeholders such as the
Financial Community and News Media
● Prior ‘false starts’ in establishing a robust BI / Data Warehousing
Program
● Key question entering 2013 - Can we launch a successful Data
Warehouse Program to enable analytics to meet these new needs?
BI Leadership Summit | © TechTarget
Going into 2013, was INTTRA Organizationally
Ready?
Yes, but …
• A clear and well communicated corporate strategy existed
• Support for BI/DW at all management levels
• Business and IT well aligned
• Forward momentum in creating a robust technical infrastructure including a somewhat under-used Data Warehouse appliance
Many Areas of Opportunities
• The information demands were not supported by a data structure optimized for on-line transaction processing
• Improvements in data quality and validation processes
• Adding skills and skilled associates to the technical team
• Reducing spreadmarts and embracing BI
• Formulating and adopting governance and standardization
BI Leadership Summit | © TechTarget
Deeper Dive to INTTRA’s Early 2013 State
BI Leadership Summit | © TechTarget
Oracle DW with some Data Marts
• Limited use
• Major performance problems
• Data silos
• Application silos
No BI Tool
• Spreadmart proliferation
• Some nice Visuals
• Excel-based
• Created with the Panopticon visualization tool
One external ‘Product’ attempt
• Web (Java) based
• Failed due to inability to justify numbers
• Difficult to maintain
Greenplum Appliance purchased
• Used sporadically based on when Oracle performance issues encountered
• Purchased with a hardware first approach
• Copy of existing mart
Data Warehousing / Analytics Principles
Implement a reusable and resilient underlying
Integration Framework
Create an integrated and comprehensive
Information Repository
leveraging the existing Greenplum Appliance
Instantiate data ordered by business Prioritized Subject
Areas
Adopt an Organizationally
Appropriate BI Tool establishing well
governed processes
Contribute Internal Business Value via
robust analytic dashboards
Devise Revenue Generating, External
Products for our customers that they
could not live without
Develop a skilled, business-savvy and
motivated BI/DW Team
BI Leadership Summit | © TechTarget
2013 DW/Analytics Roadmap
Release by Subject Area
Allow time for the robust framework to be
developed, reviewed and deployed with first
Subject Area
Load to a Unified Data Layer (UDL) via a
Staging Layer
Perform a BI Tool evaluation then
purchase, deploy and adopt the winner
Based on the selected BI tool, determine need,
approach and technique for reporting schemas /
data mart layers
Formulate and specify internal and external analytic use cases
Create and communicate BI development,
promotion, support and adoption processes
Craft a BI organization to execute and grow
BI Leadership Summit | © TechTarget
Where Are We Today?
Data Warehouse
DW Technology Established
Mostly Oracle based source systems
Oracle Data Integrator/Change Data Capture/flat files
Greenplum load utilities
Greenplum functions
5 Subject Areas in Production
Booking
Shipping instructions
Bills of Lading
Track and trace status events
Invoicing
Key Volume Metrics
160 Tables
420 Million track and trace records (400 days)
105 Million bookings (3 years)
Robust Framework
Reusable change data capture process
Includes audit and monitoring
Changes captured hourly
Processing up to 1.2 million status events a day
Unified Data From 2 booking systems
From 3 shipping instruction systems
BI Leadership Summit | © TechTarget
Where Are We Today?
Business
Intelligence
Tableau BI Tool selected, purchased, deployed and trained on
Less ‘heavy’ (including cost)
Reasons for selection ‘Visual Intelligence’ Ease of Use Enterprise Capabilities (with some annoyances)
Direct and Extract
Licensing 80 Server Users
10 Desktop Developers
Defined Iterative Development and Promotion Process
Met challenge of working around semantic layer shortcomings
Multi source access including spreadsheet and can blend
4 Tier Support: First Line Departmental Enterprise Vendor
3 Tier Development: Some Self Serve
Departmental
Enterprise
BI Leadership Summit | © TechTarget
Analytic Successes So Far
PERFORMANCE DASHBOARD
• Carrier. Customer and Sales Rep Perspective
• Key Metrics
• Container Counts
• User Locations
• Network Pairs
• Per Product, Region, etc.
• Link to Sales Force Account page
TnT DATA QUALITY SCORECARD
• Measures Completeness, Accuracy and Timeliness of Status Events
• Now know who has the best quality in Status Event delivery
• Key Learning - Certain Carriers never sent any Status Event types (e.g. Vessel Departure)
• External
• Potentially Revenue generating
TRANSPORTATION MODE SHEET
• Facilitates effort to increase Door to Door shipments
• Precisely match to Shipments
• Includes Transshipments and Inland legs
BOOKING VALIDATION REPORT
• Compared to existing DM/Spreadsheets
• Saved countless hours of SQL writing
• Holistic
• Found about 10 new DW defects
• Found problems in current reporting
BI Leadership SUMMIT | © TechTarget
Company A
Company B
Company C
Company D
Company E
Company F
Company G
Company H
Company I
Company J
Company K
Company L
Company M
Company N
Dashboard Sample
BI / Data Services Organization
● BI and Data Services are part of the Platform Services
Department
● Was mostly one man with assistance from: - Project management
- DW vendor partner
● Now expanding - Back-end / Front-end
- Transform the Business / Run the Business
- Internal / External Focus
- Looking for Tableau expertise
• Administration
• Architecture – reporting & semantic layer
• Developers
- Data Warehouse Architect
- Data Modeler
BI Leadership Summit | © TechTarget
Lessons Learned – Select the Right BI Tool
Do you really need a ‘heavy tool’?
Do you really need a large monolithic semantic layer?
Do you need to get up and running quickly?
Do you want self service and ad hoc ‘investigative’ analysis
from multiple perspectives?
Can the tool be used externally without huge cost or
complexity?
Embrace, don’t dissuade, end user report development
Extracted, self-contained analytics: take advantage of it
BI Leadership Summit | © TechTarget
Additional Lessons Learned
BI Leadership Summit | © TechTarget
Change will happen, be flexible – you don’t know everything up front
Perform robust (even if time consuming) data modeling up front including data quality analysis
Constantly test and validate and use real use cases for validating
Document - it will help create ‘end user guides’ for Analytics
Think adoption and training from day one
What’s Next
● Data Warehouse (Backend)
- Additional subject areas
- Enhancement backlog
- Operational support processes
● Analytics (Frontend)
- Increase Internal Reporting
- Embark on External Reporting
● Monitoring, Evangelizing, Marketing
● Organizational optimization BI Leadership Summit | © TechTarget
Thank You!
Follow ups? Reach me at [email protected]
Also contact and connect with me on LinkedIn at:
http://www.linkedin.com/in/strum
BI Leadership Summit | © TechTarget
Questions?