A Guide to Modern BPM Data Integration

A Guide to Modern BPM Data Integration

Kimberly Palko, Product ManagerEric Schabell, Technical EvangelistRed Hat JBoss

Agenda

• Data integration challenges with business processes

• Providing context with JBoss Middleware– JBoss Data Virtualization– JBoss BRMS– JBoss BPM Suite

• Business process data virtualization• Demo combining JBoss Data Virtualization with

JBoss BPMS

Data in disconnected enterprise data silos

NoSQL

Mainframes

Enterprise Applications

Datawarehouse and Databases

Cloud Apps

Hadoop

Files

Data Control Challenges Getting Bigger with Big Data, Cloud, and Mobile

• Security capabilities are tightly coupled to data sources• Extracting and moving data adds risk• Every project solves data access and integration in a different way• Inconsistent and decentralized control of data

BI Reports Business Process

Enterprise Applications

SOA Applications

Mobile Applications

Hadoop NoSQL Cloud Apps Data Warehouse & Databases

Mainframe XML, CSV& Excel Files

Enterprise Apps

Different security capabilities for each data source

Constant Change

Siloed &Complex

How to align?

Data Quality issues impact BPM projects

RedPoint – MDM for the Modern Data Architecture, Sept 2014

Process model complexity slows change

Point to point connections to multiple data sources inhibit agility

Business and IT both want control of the data

Business wants autonomy IT wants security and control

Consider...

How would your organization change…●If data were readily reusable in place rather than requiring significant effort to build new intermediary data tiers?●If data could be repurposed quickly into new applications and business processes?●If all applications and business processes could get all of the information needed in the form needed, where needed and when needed?

Inconsistent, Incomplete Information

Uninformed, Delayed Decisions

Costly Business Risk and Exposure

Combine process withdata in a flexible and secure way for agile business

processes

What does Data Virtualization software do?Turn Fragmented Data into Smart Data

Data Virtualization software virtually unifies data spread across various disparate sources; and makes it available to applications as a single consolidated data source.The data virtualization software implements 3 steps process to bridge data sources and data consumers:• Connect: Fast access to data from

diverse data sources • Compose: Easily create unified virtual

data models and views by combining and transforming data from multiple sources.

• Consume: Expose consistent information to data consumers in the right form thru standard data access methods.

Virtual Consolidated Data Source

BI Reports

Data Virtualization Software

• Consume• Compose• Connect

SAP Salesforce.comOracle DW XML, CSV& Excel files

Siloed & Complex

VirtualizeAbstractFederate

Easy,Real-time

InformationAccess

SOA ApplicationsDATA CONSUMERS

DATA SOURCES

JBoss Data Virtualization

DESIRED STATEData as a Service

●Standard based interface

●Single view of disparate source data

●Single point of access / integration

●Reuse of Data

But you cannot achieve this by writing more application code…

Data SourcesSiloed & Complex

Data as a Service

BI Reports Business Process

Enterprise Applications

SOA Applications

Mobile Applications

Hadoop NoSQL Cloud Apps Data Warehouse & Databases

Mainframe XML, CSV& Excel Files

Enterprise Apps

Turn Siloed Data into Actionable Information

Connect

Compose

Consume

BI Reports & AnalyticsMobile Applications

SOA Applications & Portals ESB, ETL

Native Data Connectivity

Standard based Data ProvisioningJDBC, ODBC, SOAP, REST, OData

JBos

s D

ata

Virt

ualiz

ation

Dat

a Co

nsum

ers

Dat

a So

urce

s

Design Tools

Dashboard

Optimization

Caching

Security

Metadata

Data Warehouse & Databases

XML, CSV& Excel FilesHadoop NoSQL Cloud Apps Mainframe

Enterprise Apps

Siloed & Complex

VirtualizeTransformFederate

Easy,Real-time

InformationAccess

Unified Virtual Database / Common Data ModelData Transformations

Integration of Big Data with traditional data sources

• Integrating traditional data with big data is easy

• Integration specifications can be shared or be developed for individual reports

• Caching capabilities for improved performance

MapReduce

HDFS

Hive ApplicationDatabase Server

JBoss Data Virtualization:Supported Data Sources

Enterprise RDBMS:• Oracle • IBM DB2 • Microsoft SQL Server• Sybase ASE• MySQL• PostgreSQL• Ingres

Enterprise EDW:• Teradata • Netezza • Greenplum

Hadoop:• Apache• HortonWorks• Cloudera• More coming…

Office Productivity:• Microsoft Excel • Microsoft Access• Google Spreadsheets

Specialty Data Sources:• ModeShape Repository• Mondrian• MetaMatrix• LDAP

NoSQL:• JBoss Data Grid• MongoDB • Cassandra• More coming…

Enterprise & Cloud Applications:• Salesforce.com• SAP

Technology Connectors:• Flat Files, XML Files,

XML over HTTP• SOAP Web Services• REST Web Services• OData Services

SECURITY CAPABILITIES• Authentication

– Kerberos, LDAP, SAML, SSL, WS-UsernameToken, HTTP Basic

• Authorization– Virtual data marts, Role based access

control

• Audit• Centralized audit logging and dashboard

• Administration– Centralized management of users and

groups

• Protection– Row and column masking

Optional ways to control data access using Virtual Data Marts or various views of the dataeither can be combined with security features like user role access

Data Warehouse & Databases

XML, CSV& Excel FilesHadoop NoSQL Cloud Apps Mainframe

Enterprise Apps

Base Virtual Database (VDB)

Dept1 VDB

Dept2 VDB

View2View1

Data Virtualization DesignerModel Driven Development

Eclipse-based graphical tool for • modeling, • analyzing, • Integrating,• resolving semantic

differences and • testing multiple data sources to produce • Relational, • XML and • Web Service Viewsthat expose your business data without any programming.• Shows structural

transformations and dependencies

• Defines transformations

Business Dashboard Quickly Visualize your Data

What does Data Virtualization software do?Turn Fragmented Data into Smart Data

Data Virtualization software virtually unifies data spread across various disparate sources; and makes it available to applications as a single consolidated data source.The data virtualization software implements 3 steps process to bridge data sources and data consumers:• Connect: Fast access to data from

diverse data sources • Compose: Easily create unified virtual

data models and views by combining and transforming data from multiple sources.

• Consume: Expose consistent information to data consumers in the right form thru standard data access methods.

Virtual Consolidated Data Source

BI Reports

Data Virtualization Software

• Consume• Compose• Connect

SAP Salesforce.comOracle DW XML, CSV& Excel files

Siloed & Complex

VirtualizeAbstractFederate

Easy,Real-time

InformationAccess

SOA ApplicationsDATA CONSUMERS

DATA SOURCES

JBoss BRMS and BPM

Business challenge

travel booking

review

approval

pricing

accounting

payment

final approval

Business challenge

travel booking

review

approval

pricing

accounting

payment

final approval

Visibility – difficult to know what work is where and at what stage

Agility - it takes too long to adapt the process as business needs change

Consistency – different people route work differently

Problems:

Agile Processes with BPM

BPMS

Business experts create a model of the business process

Agile Processes with BPM

A BPMS automates the process – driving claims through the organization according to the model, and managing interaction with people and applications.

BPMS

Data collection Data registration

The BPMS Solution

A BPMS tracks the status of work, providing dashboards and reports that ensure visibility into every aspect of business operations

JBoss BRMS 6 JBoss BPM Suite 6

JBoss BRMS 6EventsRules

Process

EventsRules

Business Rules Management Business Process Management

JBoss BRMS and JBoss BPM Suite

What does Data Virtualization software do?Turn Fragmented Data into Smart Data

Data Virtualization software virtually unifies data spread across various disparate sources; and makes it available to applications as a single consolidated data source.The data virtualization software implements 3 steps process to bridge data sources and data consumers:• Connect: Fast access to data from

diverse data sources • Compose: Easily create unified virtual

data models and views by combining and transforming data from multiple sources.

• Consume: Expose consistent information to data consumers in the right form thru standard data access methods.

Virtual Consolidated Data Source

BI Reports

Data Virtualization Software

• Consume• Compose• Connect

SAP Salesforce.comOracle DW XML, CSV& Excel files

Siloed & Complex

VirtualizeAbstractFederate

Easy,Real-time

InformationAccess

SOA ApplicationsDATA CONSUMERS

DATA SOURCES

Business Process Data Virtualization With JBoss Data Virtualization

and JBoss BPM Suite

Process Data Virtualization

“Process data virtualization provides virtual views of systems-of-record data in forms that process models, process applications, and users can consume while hiding the true complexity of data.”

TechRadar: Customer-centric BPM Technology, Q4 2014Clay Richardson

Complex business process using data from multiple sources

BPMS

Hadoop NoSQL Cloud Apps Data Warehouse & Databases

Mainframe XML, CSV& Excel Files

Enterprise Apps

Virtualized data layer simplifies BPM

Hadoop NoSQL Cloud Apps Data Warehouse & Databases

Mainframe XML, CSV& Excel Files

Enterprise Apps

Base Virtual Database (VDB)

BPMS

Process data virtualization Business value-add: HIGH

“The primary goal of process data virtualization is to move integration complexity out of process models and thereby support faster change of business processes. This is a crucial requirement for transitioning BPM to focus on customer engagement.

Process data virtualization also allows process changes to be pushed closer to the business, since it reduces the level of of involvement required by tech management for connecting processes and data.”

TechRadar: Customer-centric BPM Technology, Q4 2014Clay Richardson

What does Data Virtualization software do?Turn Fragmented Data into Smart Data

Data Virtualization software virtually unifies data spread across various disparate sources; and makes it available to applications as a single consolidated data source.The data virtualization software implements 3 steps process to bridge data sources and data consumers:• Connect: Fast access to data from

diverse data sources • Compose: Easily create unified virtual

data models and views by combining and transforming data from multiple sources.

• Consume: Expose consistent information to data consumers in the right form thru standard data access methods.

Virtual Consolidated Data Source

BI Reports

Data Virtualization Software

• Consume• Compose• Connect

SAP Salesforce.comOracle DW XML, CSV& Excel files

Siloed & Complex

VirtualizeAbstractFederate

Easy,Real-time

InformationAccess

SOA ApplicationsDATA CONSUMERS

DATA SOURCES

Travel Agency Demo

Insert Screen Share Here

Demo – Intro JBoss BPM Travel Agency• The project• The workflow• The rules

Demo – Acquisition integration• Data virtualization• External connection• Dashboard

Demo – Bookings registration

Data virtualization mapped to dashboard

What does Data Virtualization software do?Turn Fragmented Data into Smart Data

Data Virtualization software virtually unifies data spread across various disparate sources; and makes it available to applications as a single consolidated data source.The data virtualization software implements 3 steps process to bridge data sources and data consumers:• Connect: Fast access to data from

diverse data sources • Compose: Easily create unified virtual

data models and views by combining and transforming data from multiple sources.

• Consume: Expose consistent information to data consumers in the right form thru standard data access methods.

Virtual Consolidated Data Source

BI Reports

Data Virtualization Software

• Consume• Compose• Connect

SAP Salesforce.comOracle DW XML, CSV& Excel files

Siloed & Complex

VirtualizeAbstractFederate

Easy,Real-time

InformationAccess

SOA ApplicationsDATA CONSUMERS

DATA SOURCES

Conclusion

Benefits of Business Process Data Virtualization

• Easy access to data in real-time from multiple different sources

• Improved data quality with a single version of the truth

• More agility with simplified process models• Re-use of data integrations from multiple sources• Increased autonomy for lines of business with

departmental virtual data bases• Centralized IT control and auditing of data sources

Questions