© 2002 IBM Corporation IBM Israel Software Symposium 2003 accelerate your business DB2 Information Integrator Software Jaffa Sztejnbok IT Specialist, Information Management Global Technology Unit [email protected]
Dec 14, 2015
© 2002 IBM Corporation
IBM Israel Software Symposium 2003
accelerate your business
DB2 InformationIntegrator Software
Jaffa SztejnbokIT Specialist, Information ManagementGlobal Technology [email protected]
accelerate your business
IBM Israel Software Symposium 2003 2
Agenda
What is Enterprise Information Integration
Without Information Integrator
Data Challenges
Complementary Information Integration Approaches
IBM DB2 Information Integration Products and Value
IBM’s Information Integrator 8.1
Demo
accelerate your business
IBM Israel Software Symposium 2003 3
What is Enterprise Information Integration ?
Provides access to diverse, distributed, and real-time data as if it were a single source, no matter where it resides.
Helps businesses : –Shorten application development time
–Improve productivity and application efficiency
–Leverage existing data assets for the benefit of the business
accelerate your business
IBM Israel Software Symposium 2003 4
Without Data Federation
DB2
Oracle
SQLServer
Excelspreadsheet
XMLdocument
InternetSSL
clientapplications
Browsers
Data managementlayer
Applicationlayer
Webservers
accelerate your business
IBM Israel Software Symposium 2003 5
New composite applications need data from multiple sourcesƒ Consumers expect holistic, personalized, and value-added contentƒ Relational, XML, packaged applications, content repositories, file
systems all contain critical business information
Increasing emphasis on current dataƒ Real-time analyticsƒ Business activity monitoring
Petabytes will be the measure of available online data ƒ All client interactions are important ( e.g. instant messages, audio
records, Web traffic,…)
ƒ Internet and intranet content
Variety, Velocity, and Volume
Data Challenges
accelerate your business
IBM Israel Software Symposium 2003 6
Complementary Information Integration Approaches Consolidate data for local access
– Data warehouses
– Operational data stores
– Production applications
– Creating additional reference copies
– Typically managed by ETL (Extract, Transform, Load) or replication technologies
Integrated access to distributed sources, Distributed Access
– Real time data, e.g., stock quotes ,Extending a data warehouse with real-time data
– Data changes rapidly
– Wide heterogeneity in data to be accessed ,relational and non-relational format
– Data which is not practical or possible to copy and when movement of data is small
accelerate your business
IBM Israel Software Symposium 2003 7
BiologicalBiologicalData and Data and
AlgorithmsAlgorithms
relational databases
data warehouses
DataSources
Extended Search
Sources
spreadsheets
contentrepositories
…@office e-mail reports fax
ContentSources
IBM DB2 Information Integration Products
DB2 Information Integrator
SQL programming model Leverage SQL skills and
tools Federated data server
and replication server
DB2 Information Integrator for Content Content programming model Leverage CM skills and tools Federated data server, text
mining, and workflow engine
accelerate your business
IBM Israel Software Symposium 2003 8
DB2 Information Integrator 8.1A Federated Data Server – Query distributed data as if it where a single source
Define integrated view across diverse and distributed data– Wide range of data and content sources
– Extensible to virtually any data source
Query as if a single source– Use standard SQL query and SQL expressions
– Include text semantics in the search
– Surface specialized functions into SQL
– Leverage query optimization and caching
Compose XML documents– Combine diverse sources
– Validate against DTDs or schema
Publish results to a message queue– Familiar DB programming model
Single source, relational updates
Integrated SQLView
DB2, Oracle, SQL Server, Sybase, Teradata, OLE DB, ODBC, Excel, XML, message queues, Web services, flat files, document repositories, content repositories, LDAP directories, WWW,
email databases, and more.
accelerate your business
IBM Israel Software Symposium 2003 9
DB2 Information Integrator 8.1
A Replication Server – Manage consolidation for performance and availability
Distribute data among relational databases
– DB2, Informix, Microsoft, Oracle, Sybase, Teradata
Support flexible topologies– Distribution: One to many
– Consolidation: Many to one
Match data movement modes to usage requirements– Table-at-a-time for warehouse loading during
batch window
– Transaction-consistent for online data Choose latency characteristics
– Scheduled, interval-based, continuous
Apply transformations in-line – Standard SQL expressions or stored procedure execution.
DB2
Microsoft Microsoft
accelerate your business
IBM Israel Software Symposium 2003 10
Define integrated views across diverse and distributed data
ƒ IBM Content Manager portfolio and other content repositories e.g. FileNET, Lotus
databases, ODBC and JDBC compliant relational databases, and IBM Lotus
Extended Search sources (LDAP directories, WWW, email databases,…)
Search federated data
ƒ Search application uses the IBM Content
Manager API
Mine additional metadata from text
documents
ƒ Identify document language
ƒ Extract entities like names or technical terms
ƒ Categorize documents based on a taxonomy
ƒ Group documents based on related content
ƒ Create a document synopsis
Define workflows
DB2 Information Integrator for Content
accelerate your business
IBM Israel Software Symposium 2003 11
DB2 Information Integrator Value Extend current investments
– Work within your existing infrastructure
– Consolidate data or access distributed data as if it were a single data source
– Combine existing data and content assets in new ways
– Use familiar SQL programming model and existing tools
– Build on a standards-based, strategic integration platform
Speed time to value for composite applications– Reduce hand-coding 40%-65%
– Reduce skill requirements
– Reduce development time by half
Control costs– Reduce payroll costs
– Reduce need to rip and replace
– Reduce need to manage redundant data
accelerate your business
IBM Israel Software Symposium 2003 12
ApplicationDeveloper
Speeding Application Development
RDBMS
Non-relational data
Non-traditional data
Development effort to handle:
Unique interfaces for each data type
Joining data from varied sources
Aggregation and grouping
Correlating dataSpecial features:Set processingIn-built db
transformation functions
OptimisationAutomatic local
cachingData driven triggers
OtherSQL is on OPEN StandardSQL is easily testable,
independent of the application
JDBC, XML, WebServices
II handles:Interfaces for each
data typeJoining data from
varied sourcesTransformationCorrelating data
accelerate your business
IBM Israel Software Symposium 2003 13
Crystal Decisions
Vision As a world-leading information infrastructure company, Crystal Decisions helps businesses make better decisions by bringing together their people and their information.
Challenge Improve response time for complex queries over distributed
heterogeneous data sourcesSolution
Provides transparent, globally optimized access to heterogeneous, distributed data. Crystal Reports accesses the distributed data as if it were a single database. Response time improvement of up to 98% seen in house.
Business Value"Users of Crystal Reports and Crystal Enterprise, with DB2 Information Integrator, can … discover new ways to meet the information needs of their organization."
• Janet Wood, Vice President of Business Development, Crystal Decisions.Competitive Value
“DB2 Information Integrator provides Crystal Reports with exceptionally fast and efficient federated querying capability.“
• Trevor Smith, Program Manager, Business Development Group, Crystal Decisions
accelerate your business
IBM Israel Software Symposium 2003 14
Without Data Federation
DB2
Oracle
SQLServer
Excelspreadsheet
XMLdocument
InternetSSL
clientapplications
Browsers
Data managementlayer
Applicationlayer
Webservers
accelerate your business
IBM Israel Software Symposium 2003 15
Federated Access to Diverse Data
accelerate your business
IBM Israel Software Symposium 2003 16
Information Integration
IBM DB2 Information Integrator Software
Data federation–Extensible read/write access across diverse data and content sources
–Database programming model (SQL)
–Content programming model (OO API)
Data placement –Caching and replication over heterogeneous information
Data transformation –SQL, XML, Web services
–Advanced search and mining
–Metadata management
Part of a complete integration solution –XML publishing, consumption, and interchange
–WebSphere business integration
–Open platform based on industry standards
Integrating diverse business information
across and beyond the enterprise
accelerate your business
IBM Israel Software Symposium 2003 17
Functions
Data Federation Transparency: hides differences among sources
ƒ Appears to be one source
ƒ Supports a high level query language
ƒ Functional compensation and passthru
Heterogeneity: integrates data from diverse sourcesƒ Relational, XML, flat files, spreadsheet,
messages, content repositories, Web, …
High Functionƒ One query integrates data from multiple sources
ƒ Capabilities of sources as well
Extensibilityƒ Access wide range of data sources
ƒ Development wrapper toolkit
Autonomyƒ Non-disruptive to data sources, existing applications, systems.
accelerate your business
IBM Israel Software Symposium 2003 18
Federation leverages a full database engineƒ Query Processor, Execution Engine, Catalog, Client Access, Security,
Transactions
Query processing extended for Federated Dataƒ Pushdown Analysisƒ Analyze how to decompose a user queryƒ Generate an optimal query execution plan using cost estimates including
data source knowledge: database statistics, indexes, source functions, server and network capacities
ƒ Allows function compensation
Performance , Optimization of distributed queries
accelerate your business
IBM Israel Software Symposium 2003 19
Replication Architecture
accelerate your business
IBM Israel Software Symposium 2003 20
Improve query performance and availability Administrator defines Materialized Query Table
ƒ Precomputed or frequently used valuesƒ Any data from the federated systemƒ Application indicates ability to use cacheƒ Implicit or explicit use
Developer enables cache useƒ If enabled, reads are handled from the
cache, writes passed through to the sourceƒ If not, reads and writes passed through to
sourceCache refresh managed:
ƒ Manuallyƒ By replication
Flexible caching topologies supported
Heterogeneous Caching Feature
accelerate your business
IBM Israel Software Symposium 2003 21
Four important tasks:ƒ Data modeling
– Map data model to relational data model (tables with rows and columns)
– Map functions into SQL operationsƒ Query Planning
– Represent data source capabilities– Push down as much work to data source as sensible– Detect missing function at source (so engine can compensate)– Supply cost and cardinality information
ƒ Connection and transaction management ƒ Query Execution and data retrieval
– Execute parts of a user’s query for a specific data source
Wrappers
accelerate your business
IBM Israel Software Symposium 2003 22
Configuration steps:
ƒ Wrapper: the wrapper code module itself
ƒ Server: a specific data source, with associated attributes
ƒ User mapping: information needed to connect to a specific
server
ƒ Nickname: a specific data set managed by a server,
mapped to rows and columns in the federated server
Defined to system via DDL commands
ƒ GUI administration generates DDL
Stored in the system catalog
Configuration
accelerate your business
IBM Israel Software Symposium 2003 23
Administration Tools
accelerate your business
IBM Israel Software Symposium 2003 24
SybaseOracleSQL ServerDB2InformixODBCTeradata
Wrappers which support discoveryHMMEREntrezXMLFlat FileExcelExtended Search
"Create Nicknames" window
Launches customized GUI
Returns Nickname defintions
Customized "Discover" GUI
Discovery for Nicknames
accelerate your business
IBM Israel Software Symposium 2003 25
Definitionsƒ Manage control definitions
for replicationƒ Customize names and sizes
of objectsOperations
ƒ Start Capture, Apply, Monitor, Analyzer, and Trace
ƒ Issue commands such as STOP or STATUS
Monitoringƒ Perform static and dynamic
monitoring
Replication Administration
accelerate your business
IBM Israel Software Symposium 2003 26
Application Development : Access DB2 catalogs and DB2 II federated sources
DB2 Development CenterWebsphere Studio
Microsoft Visual Studio .NET
accelerate your business
IBM Israel Software Symposium 2003 27
Demo…
Query Result
ExcelSQL Server
DB2
accelerate your business
IBM Israel Software Symposium 2003 28
For more information
http://db2ii.dfw.ibm.com/demo/
http://db2ii.dfw.ibm.com/demo/
accelerate your business
IBM Israel Software Symposium 2003 29
Summary Information integration is a foundation for companies to build an On Demand Operating
Environment enabling them to align their IT infrastructure to business priorities
DB2 Information Integrator provides access to diverse, distributed, and real-time data as if it were a single source, no matter where it resides.
DB2 Information Integrator will help businesses – Shorten application development time
– Improve productivity and application efficiency
Rely on IBM’s proven technology and support for open standards
accelerate your business
IBM Israel Software Symposium 2003 30
The whole is worth more then its constituent parts
DB2 Information Integrator
helps businesses to leverage existing data assets into knowledge
for the benefit of the business
accelerate your business
IBM Israel Software Symposium 2003 31
Don’t forget to give us feedback
Presentation Code: A4