Metadata Overview Deck - IBM Overview Deck - IBM
Post on 27-Apr-2018
224 Views
Preview:
Transcript
© 2008 IBM Corporation
Last Updated 9-8-08
InfoSphere Metadata Comprehensive Deck
Information Management Software
Agenda
• 2008 Metadata Market Survey
• InfoSphere Information Server Metadata Approach
• InfoSphere Business Glossary
• InfoSphere Metadata Workbench
• Third Party Metadata
• Cognos Integration
• Case Studies
© 2008 IBM Corporation
2008 Metadata Market Survey
IBM and Gavilan Research Associates (GRA)
Information Management Software
Top 5 areas companies said they need to improve their metadata:• Documenting business metadata (business rules & element descriptions) -
80.8 % of respondents
• Support for Data Governance and Data Stewardship (67.5% of respondents)
• Documenting a Data Warehouse, Data Mart, or Analytical Application (64.8% of respondents)
• Support for Data Lineage reporting and/or Data Mapping analysis (64.9%)
• Implementing a corporate Data Dictionary or Data Glossary (64.7%)
Of those surveyed:Less than 7% said their implementation was highly successful !
IBM & Gavilan Research Associates 2008 Metadata Survey – 287 respondents
IBM and Gavilan 2008 Survey Highlights: Turning your Data into Information with Metadata
Information Management Software
What Are Your Top Metadata Management Issues ?
Respondents could choose multiple answers, which created a broad spectrum.
The “top 5” metadata management issues in 2008 are:
• documenting business metadata (business rules & element descriptions) - 80.8 % of respondents
• support for Data Governance and Data Stewardship (67.5% of respondents)
• documenting a Data Warehouse, Data Mart, or Analytical Application (64.8% of respondents)
• support for Data Lineage reporting and/or Data Mapping analysis (64.9%)
• implementing a corporate Data Dictionary or Data Glossary (64.7%)
Respondent Comments:
• “My top issue is getting metadata out of existing toolsets into a repository, where it can be accessed and reported on.”
• “Change Management is our biggest issue! How could you miss this point?”
Source: IBM / Gavilan Research Associates 2008 MetaData Market Survey
© 2008 IBM Corporation
InfoSphere Information Server Metadata Approach
Information Management Software
So what exactly is Metadata?
Metadata enables you to put context and meaning to things.
It is generated and consumed by every organization and software product.
Made by
Since
Type of food
With a special
ingredientWith many
varieties
Stored in a can
With a trademark label
Information Management Software
Metadata Primer for Business
• Business Metadata• Business rules, Stewardship, Business Definitions, Auditing Terminology,
Glossaries, Algorithms and Lineage using business language. Audience: Business users.
• Technical Metadata• Defines Source and Target systems, their Table and Fields structures and
attributes, Documentation for Auditing Derivations and Dependencies. Audience: Specific Tool Users – BI, ETL, Profiling, Modeling.
• Operational Metadata• Information about application runs: their frequency, record counts,
component by component analysis and other statistics for auditing purposes. Audience: Operations, Management and Business Users.
B
T
O
Literally, “data about data” that describes your company’s information from both a business and a technical perspective
Information Management Software
MetaData is what describes, defines, & inter-relates your key information assets
• “Social Security Number: a unique identification number issued to citizens, permanent residents, and temporary (working) residents by the US Government ...”
• pic XXX-XX-XXXX
• “Profit Amount: a currency value that is calculated by combining data from the Customer Master database and Wholesale Inventory applications . . .”
• Calculation included on Monthly Report
Let’s put Metadata to an Example
Suppose you have these items of “data”
• 552-90-3456
• $233,567.00
Information Management Software
• What data or information exists ?
• Where is it being used ?
• What is its business definition ?
• What other names has it been called or is being called ?
• How is it inter-related to other information ?
• Who is using it ?
• Why do we need it ?
• When was it last updated ?
Metadata helps answer important questions such as:
What is “Profit Amount” ?How is it defined ?
How is it calculated ?Where is it stored or used ?Is it reliable ? Accurate ?
If I make a change to “Profit Amount” – what will be
impacted ?
Information Management Software
Metadata Business Drivers
What is happening in the Data Integration Market that requires Metadata?
Governance and Compliance Regulations are increasing
- How do organizations comply and meet documentation requirements?- How can organizations ensure accountability and responsibility?
Business Competition continues to grow
- How do organizations individualize their customer experience?- How can organizations get access to information to make correct decisions?
Costs and system complexities are expanding
- How can organizations drive optimization with integration?- How do organizations manage complex software environments?
Information Management SoftwareIBM Data Governance Maturity Model - measurable results
Business Transformation
Information Management Software
13
1990
2008
First Metadata RepositoriesIBM Data Dictionary
Single EnterpriseRepositoryIBM led AD Cycle
DataWarehouseRepositoriesETL, DQ, DPe.g. Ascential Metastage
1980
2000
1970
2007
2002Federated RepositoriesAnd Registriese.g. Unicorn
Understand how software and data services related to business processes workflows, and the requirement to publish, reuse and govern these services.
Steve Born
Metadata ConsolidationSolutions
Community FocusedRepositories… IBM buys Ascential
IBM buys Unicorn
less-ambitious more-scoped metadata
management solutionsConsolidation to central repository using
parsers, bridges and generators
e.g. Dovetail
1/3 failed miserably, 1/3 succeeded marginally, 1/3 demonstrated good ROI
$50 million to $100 million mainly maintenance
and most organizations decided live with metadata definitions in multiple
technologies, and manually synchronize
Information Server LaunchedA Brief History of Metadata – Where did we come from?
Information Management Software
InfoSphere Information
Server MetadataIBM Clean, controlled metadata supply
Where IBM Fits in the Metadata Life Cycle
Enterprise Repositories
Information Management Software
Why Businesses Care
Information Management Software
Metadata Business ChallengesMetadata is naturally a very complex subject which virtually all organizations address at some point and time. Some successfully and some unsuccessfully.Key challenges:
• Obtaining agreement on what metadata means• What does metadata mean to a specific organization or division?• What metadata is important to track and manage?• How does each group use metadata for their particular job?
• Selecting the correct metadata strategy for particular business requirements• How many and what kinds of silos of metadata exist today in
organization? • Are there revenue $ at risk, compliance issues, regulatory rules which
must be addressed?• Do we have the flexibility to assess the impact of changes with our
current architecture?
• Promoting adoption of a metadata strategy and associated technology• How does our approach address our different user’s needs?• Is our approach one that easy to use and facilitates adoption rather
than hinders it?• Do we have business and technical sponsorship?
Information Management Software
Understand Cleanse Transform Deliver
Parallel ProcessingRich Connectivity to Applications, Data, and
Content
IBM InfoSphere Information Server
Discover, model, define, and govern
information structure and
content
Standardize, merge,and correct information
Combine and restructure
information for new uses
Synchronize, virtualize and move information for in-
line delivery
Unified Deployment
Unified Metadata Management
Delivering information you can trust
The IBM Solution: InfoSphere Information Server
Information Management Software
InfoSphere Information Server: Unified Metadata Management
Store Enhance Exchange
Active, centrally managed repository with secure access via services layer
Share and deliver relevant information across the
organization
Define relationships, control extensibility, and link 3rd party metadata
BusinessUnified Metadata Management| |Technical Operational
Any User
• Data integration metadata repository – core to InfoSphere Information Server
• Facilitates change management & reuse
• Simplifies integration
• Increases trust and confidence in information
• Enable data governance initiatives
Information Management Software
Store Enhance
Active, centrally managed repository with secure access via services layer
Define relationships, control extensibility, and link 3rd party metadata
BusinessUnified Metadata Management| |Technical Operational
DevelopersSubject Matter Experts
DataAnalysts
Business Users
Architects DBAs
Enhance Collaboration & Alignment
Role-based Tools with Integrated Metadata
Exchange
Share and deliver relevant information across the
organization
InfoSphere Information Server: Unified Metadata Management
© 2008 IBM Corporation
InfoSphere Business Glossary
Information Management Software
Business Glossary Overview
• Applications
• Users
• Business Glossary
• Business Glossary Anywhere
Information Management Software
Applications of a Business Glossary
Simply put, a Business Glossary is created to represent the language of the business, independent of technology
Three Primary Applications:Three Primary Applications:
1.1. OwnershipOwnership Identifying stewardsIdentifying stewards
Managing contentManaging content
2.2. CollaborationCollaboration Common, approved vocabularyCommon, approved vocabulary
Sharing domain expertise - Business & ITSharing domain expertise - Business & IT
3.3. AuditabilityAuditability Evolution of languageEvolution of language
Centralized managementCentralized management
All key enablers to regulatory compliance and All key enablers to regulatory compliance and support the IBM Data Governance Maturity Modelsupport the IBM Data Governance Maturity Model
Information Management Software
Business Glossary: For Every Enterprise User
• General Business Users
• Business Analysts
• Data Analysts
• BI and ETL Developers
• Data Stewards
• Subject Matter Experts
• Business Glossary Authors
• Business Glossary Administrators
Examples of most common users include:Examples of most common users include:
Information Management Software
Understanding the value of Business Metadata• In the language of the business, independent of
technology• Documents the business meaning of data & related
technology assets• Used to
• define a shared meaning• standardize names• establish responsibility, accountability, and traceability• govern access• share insights & experiences among users• represent business hierarchies• document business descriptions, examples,
abbreviations and synonyms
• Must be managed by those that understand the meaning and importance of the information assets to the business
• Better aligns the efforts of IT with the goals of the business
Information Management Software
InfoSphere Business GlossaryCreate and manage business vocabulary and relationships
Steward
Features
• Facilitate business & IT communications by creating & managing a common business vocabulary
• Web based interface shared across enterprise business teams
• Allows creation of stewards & assignment of their responsibilities for terms & assets.
• Link business terms to information assets
Benefits
• Aligns the efforts of IT with the goals of the business • Provides business context to information technology
assets• Establishes responsibility and accountability in
accordance with data governance policies
Steward Console
Business Users
Business Interface
Information Management Software
Populate: Import IBM Industry Models
Jump-start your Business Glossary with thousands of industry standard terms from the IBM’s Industry Models. Refined by real customer usage, leverage years of
experience and expertise to provide the best foundation for your glossary.
Insurance Healthcare
TelcoIndustrial Retail
Financial Services
Available now!Available now!
Information Management Software
Populate: Import InfoSphere MDM for PIM
Share your product category information structures from
Master Data Management with Business Glossary enabling
access by all enterprise users.
Information Management Software
Populate: Create and Assign Stewards
• Create and Assign Stewards to Terms, Categories or Assets
• Who is responsible for this Term?
• Who is responsible for this Category?
• Who is responsible for this Asset?
• View Contact Information for Steward
Information Management Software
I can set this up so that my people immediately see the most used
business hierarchies
I didn’t need any special training to use the business glossary. It’s laid-
out exactly as I expected.
Access: Simple to use – business browser
Information Management Software
Real-time access to business glossary from any desktop application
Features• Click on a term and view definition in a pop-up
window without loss of context or focus• Intelligent matching returns best candidates in a
single search
• Search engine for terms and categories
• Access steward contact information directly
• Security enforced via the Information Server common security layer
Benefits• Increased trust and acceptance of information by
delivering definitions in context• Expanded adoption of enterprise glossary outside of
Information Platform technologies• Improved information availability with multiple access
mechanisms for electronically stored information (ESI)
ANY User
From Any Application...
Pop the Definition!
InfoSphere Business Glossary Anywhere
Information Management Software
Business Glossary Anywhere with Cognos
1. Highlight Term Active and click to look up in Business
Glossary
2. Candidate Terms returned.
3. Select Term to view full details.
Information Management Software
Shared Information Server metadata repository
Category: CostsTerm: Tax ExpenseFull Name: Tax to be paid on Gross Income“The expense due to taxes …..”(John Walsh is responsible for updates. 90% reliable source)Status: CURRENT
Database = DB2
Schema = NAACCT
Table = DLYTRANS
Column = TAXVL
data type = Decimal (14,2)
Derivation: SUM(TRNTXAMT)
Achieve a common vocabulary between business & technical users!
InfoSphere DataStage InfoSphere Business Glossary
Manage: Business and IT understanding
33
Information Management Software
Business Glossary Business Benefits
• Enables data governance• Common language supports compliance regulations such as Basel II• Represent and expose business relationships
• Accountability and responsibility• Assign stewards as single point of contact for control
• Improved productivity• Administrators can tailor the tool to the needs of their business users • Access information you need when you need it
• Increased collaboration• Capture and share annotations between team members• Greater understanding of the context of information.
• Greater trust in information• Immediate, in-context access to terms and definitions, improves decision making
© 2008 IBM Corporation
InfoSphere Metadata Workbench
Information Management Software
Metadata Workbench Overview
• Applications
• Users
• Metadata Workbench
Information Management Software
Applications of Metadata WorkbenchSimply put, Metadata Workbench allows you to understand where
information came from and where it is used.
Three Primary Applications:Three Primary Applications:
1.1. TrustTrust Data lineage for traceabilityData lineage for traceability
Understand relationshipsUnderstand relationships
2.2. ProductivityProductivity Assess impact of changeAssess impact of change
Maximize re-use Maximize re-use
3.3. GovernanceGovernance Visibility and reportingVisibility and reporting
Centralized managementCentralized management
All key enablers to regulatory compliance and All key enablers to regulatory compliance and the IBM Data Governance Maturity Modelthe IBM Data Governance Maturity Model
Information Management Software
Metadata Workbench: For Projects leads & Architects
• Project Leaders
• Data Architects and Modelers
• DataStage Developers
• BI and OLAP Developers
• Business Subject Matter Experts
• Business Users and Compliance Officers
Examples of most common users include:Examples of most common users include:
Information Management Software
InfoSphere Metadata Workbench Exploration of assets generated and used by Information Server
Features
• Cross-suite view and administration of Information Server assets
• Explore, analyze and manage metadata graphically and textually
• Comprehensive reporting on data lineage
• Assess dependencies across Information Server & 3rd party tools
• Perform searches and build customized queries
Benefits
• Assess and mitigate change management risk
• Support compliance and governance initiatives requiring data lineage (eg Sarbanes-Oxley, Basel II)
• Proactively manage and administer complex data integration environments
IT DevelopersAdministrators
Project Managers
& DBAs
Information Management Software
Information Server: Optimizing Application DevelopmentImport Industry
Data Models
Search and Profile Source Data
Information AnalyzerBusiness GlossaryDataStage and QualityStage
Transform and Cleanse
Map Sources to Target Model
FastTrack
Services Oriented Architecture
Information Services Director
Populate
Share
Deploy
Share Share Share
LinkDefi
ne
Create
Unified metadata for active administration, management and reporting Metadata Workbench offers complete visibility and control of
metadata
Exchange Data Structures
Link
Common Enterprise Vocabulary
Rational Data Architect
Information Management Software
Metadata Workbench Integration Asset Categories
Information Management Software
Metadata Workbench Feature Overview
EXPLORE
Explore key Integration Assets:
Jobs, Reports, Databases, Models, Terms, Stewards, Systems, Specifications, Quality Rules
Easy navigation of key Integration Assets
Simple and advanced search
Integrated cross-view of Information Server and 3rd party assets
Graphical view of Asset Relationships
ANALYZE
Trace data movement to and from databases, jobs and reports for full lineage
Understand business meaning of columns, tables, and other assets
Assess the impact of change across Integration assets
Robust query builder
Analyze dependencies and relationships between key Integration assets, Business Intelligence Reports and data models
ADMINISTER
Manage Integration Assets to enable in-depth analysis
Assign security roles
Link together multiple viewpoints of design assets from ETL, business, BI and modeling with operational metadata
Edit names and descriptions of Integration Assets
Access runtime information to enrich reporting
Import export manager for 3rd party integration
Information Management Software
Explore: Homepage
The homepage offers direct access to Discover key Information Assets, Find any Information Asset or execute Queries.
Information Management Software
Explore: Graphically Browse
Browse Database, Files, DataStage/QualityStage Jobs,
Table Definitions or other objects within their Folder and
Project placement.
Flexible browsing capabilities enable users to find information
easier and better manage complex environments
Information Management Software
Explore: Textually Search Where and how do I find out details about assets?
• Full search and explore capabilities
• View in textual list or graphical format
• Search multiple ways including by asset class, name or by asset property
Information Management Software
Analyze: Data Lineage Where does a Field of Data in this Report Come From?
Source TablesIBM Information Server
• Import & Browse Full BI Report Metadata
• Navigate through report attributes
• Visually navigate through data lineage across tools
• Combines operational & design viewpoint
Information Management Software
Analyze: Cross-Tool Impact Analysis Where Happens if I Change this Column?
• Show complete change impact in graphical or list form
• Includes impact on reports in BI tools
• Visually navigate through impacted objects across tools
• Allows impact analysis on any object type
Information Management Software
Analyze: Business MeaningWhat does this Field Mean?
• Show relationships between business terms, data model entities, and technical and report fields
• Provides cross-tool mapping of business terms
• Allows field meaning to be understood
• Allows business term relationships to be understood
Information Management Software
Metadata Workbench Business Benefit
• Good exploitation and coordination of metadata across tools is very compelling
• Tool Integration & metadata collaboration saves time and money, and improves Quality of results.• An organization’s ability to Govern Data is significantly improved.• A organization is better able to Manage Change, more Agile.• A Shared, Common, Vocabulary saves time and effort, helps development, and makes data more
accessible & more understandable.
The key is making the metadata capture and share process effortless
© 2008 IBM Corporation
Third Party Metadata
Information Management Software
Import Export Manager for Information Server
Expand visibility of metadata touch-points in support of data integration projects
Features• Security enforced via Information Server common security
layer as well as the 3rd party application security layer
• Metadata Bridges interchange metadata with each specific application a consist of a model, a decoder, and an encoder which require no coding.
• Import capabilities for 3rd party BI tools (Cognos, Business Objects, MicroStrategy), data modeling tools (ERwin, RDA) and databases (ODBC connections to all major RDBMS)
• Support a variety of import formats including XMI, XML, UML, CWM and CSV metadata exchange formats
Benefits• No manual interface coding required for 3rd party
metadata visibility• Visibility of data modeling to ETL to report layer
minimizes risks of overlooking critical dependencies• Leverage common metadata exchange environment for
application development consistency
IT Developers
IT Administrators
Information Management Software
Meta Integration Technologies, Inc. (MITI)
• OEM of 3rd party metadata bridges for import- More than a dozen major vendors OEM MITI bridges*
• IBM and MITI jointly certified and tested bridges
• Additional (MITI) bridges
- Many bridges are available “as-is” and can be easily enabled post installation.
* http://www.metaintegration.net/Partners/Directory.html
Information Management Software
Fully Supported Bridges
IBM and MITI both perform QA testing independently
As of 8.1 releaseAs of 8.1 release
Information Management Software
Bridges Provided As-Is
MITI performs QA testing independently
As of 8.1 releaseAs of 8.1 release
Information Management Software
Bridges Provided As-Is (continued)
MITI performs QA testing independently
As of 8.1 releaseAs of 8.1 release
Information Management Software
ETL Operational Metadata (Job Run Information)
Physical Schemas
ETL Job Design
BI Reports
Business Glossary & IS Users
The Areas of Metadata
Information Management Software
ETL Operational Metadata (Job Run Information)
Physical Schemas
ETL Job Design
BI Reports
The Areas of Metadata Connected
Business Glossary & IS Users
© 2008 IBM Corporation
Case Studies
Information Management Software
Melbourne Health (MH)Advances clinical research with next generation genetic information infrastructure
• Melbourne Health’s legacy I/T infrastructure had limited ability to access and share medical data.
• Their research data, diagnostic results, and clinical documentation were stored in mixed formats across mixed environments: MS SQL Server, MS Excel and MS Access
• They managed disparate information silos without being able to gain insight and meaning from them – key to unlocking the advancement of medical research and improving treatments.
Challenge
• Melbourne Health is deploying their BioGrid Australia project in 3 phases from 2003-2009 in collaboration with hospitals and research institutes.
• The project utilizes the following key software:
• IBM DB2
• IBM InfoSphere Information Server: InfoSphere Business Glossary, IBM InfoSphere DataStage, and IBM InfoSphere Federation Server
• SAS Enterprise Business Intelligence Server, SAS Web Report Studio, and SAS Enterprise Guide
Solution
Benefits• The BioGrid Austalia virtual platform currently links
research databases across 16 organizations. A world’s first and milestone achievement for the research community.
• BioGrid provides access to a comprehensive set of research data which ultimately leads to improved research accuracy and treatment quality, by:
• Linking data across disparate clinical and biomedical data sources within and across institutions
• Linking patient/subject records by assigning data Unique Subject Identifiers (USI’s)
• Providing a uniform interface and access mechanism
• BioGrid is a building block of a larger national grid initiative to share research data for the priority disease group of cancer.
APPROVED FOR EXTERNAL USE
Information Management Software
Melbourne HealthScreenshot: BioGrid Australia
APPROVED FOR EXTERNAL USE
Information Management Software
Melbourne HealthClient Quotes
"At this stage, our researchers are already able to carry out previously impossible studies to stratify patients by genetic types. Researchers are already seeing the initial fruits of collaboration, which will stimulate further cooperation as BioGrid is expanded. Ultimately, BioGrid equips us with capabilities to accelerate clinical research through collaboration.“
– Robert Merriel, Director of Business Development, Melbourne Health
“IBM InfoSphere Information Server is a key toolset for our multi-phase, multi-year Molecular Medicine Informatics Model project. We are leveraging the capabilities of InfoSphere Business Glossary, a module of IBM InfoSphere Information Server, to share information about the business and technical metadata stored in our research repositories so that users can have the relevant information at their fingertips.”
– Robert Merriel, Director of Business Development, Melbourne Health
APPROVED FOR EXTERNAL USE
Information Management Software
Business challenge Business intelligence reports are used to make some of this company's important business decisions. Understanding how these reports are created to ensure validity is a must have requirement. Today, when errors or questions about reports come up, a manual , time consuming, and error prone discovery process is completed by DataStage Developers to track the lineage of the BI report field in question.
SolutionAs a long-time DataStage user, this Customer saw immediate value in leveraging the Metadata Workbench to better understand lineage and impact analysis across thousands of DataStage jobs. The capabilities of the Metadata Workbench could be used in place of tedious and error prone documentation currently used to trouble shoot errors or questions about business intelligence reports. In addition, the creators of BI reports saw value in accessing the Metadata Workbench on their own to do their own research about reports in question.
Benefits Improved TRUST in BI reports
Time saved to do manual troubleshooting of errors in BI reports
Automated the documentation process
“The Metadata Workbench helps this customer prove through lineage and impact analysis that their BI reports contain Trusted Information.”
—Desiree Ruiz-Amirgholi IBM Lab Advocate and Product Manager for the Metadata Workbench
Major Insurance Company in the UK – Metadata WorkbenchImproved Trust of BI Reporting Through Lineage and Impact Analysis
© 2008 IBM Corporation
Summary
Information Management Software
Information Server Metadata Differentiators
Auditing and traceability - single data integration repository
Enhanced collaboration – seamlessly share information across user roles
Simplified development – reduce manual steps with integrated platform development
Streamlined information access – promote enterprise adoption through ease of use
Strategic integration with IBM brands – roadmap for success
top related