Monitoring Your Business with WSO2 BAM Anjana Fernando Senior Technical Lead WSO2 Inc. Chathura Ekanayake Senior Technical Lead WSO2 Inc.
Jun 14, 2015
Monitoring Your Business with WSO2 BAM
Anjana FernandoSenior Technical Lead
WSO2 Inc.
Chathura EkanayakeSenior Technical Lead
WSO2 Inc.
Outline• Introduction to Business Activity Monitoring
• WSO2 Business Activity Monitor
• WSO2 BAM Components
• WSO2 BAM Toolboxes
• WSO2 BAM High Availability Distributed Deployment
• Demonstrations
Business Activity Monitoring
“The aggregation, analysis, and presentation of real-time information about activities inside organizations and involving customers and partners” - Gartner
Aggregation
● Capturing data
● Data storage
● What data to capture?
Analysis● Data operations
● Building KPIs
● Operate on large amounts of historic data or new data
● Building BI
Presentation
● Visualizing KPIs/BI
● Custom Dashboards
● Visualization tools
● Not just dashboards!
WSO2 Business Activity Monitor
• Fully-open source solution for aggregating and analyzing data and presenting information about business activities.
• Cloud-enabled, lightweight, developer-friendly and easy-to-deploy
• High performance data capture framework
• Elastically-scalable data analysis powered by Apache Hadoop
• Pre-built Data Agents for WSO2 products
Architecture
Data Agents
Data Agents● Compatible with CEP/BAM
● Get data across to BAM
− Service monitoring feature – WSO2 AS, DSS, ESB, API Manager
− Mediation monitoring feature – BAM Mediator for WSO2 ESB
− Custom data-agents
● Asynchronous & non-blocking
● Thrift for high performance message throughput
Custom Agents
• Data agents are meant to be installed/plugged into the server to be monitored
• A Java SDK is provided
• Through Thrift, different languages can be supported
Data Receiver
Data Receiver● Receives data and stores it in Cassandra
○ Scalable, big data repository
● Asynchronous & non-blocking
○ Combination of Cassandra, Thrift and the non-blocking nature results in extremely fast writes
● Shared with WSO2 CEP for real time analysis
● Supports Thrift & REST API
● Supports plugging in of different receiver types
Data Model• Data is sent over using strongly typed Data Streams
{ 'name':'phone.retail.shop', 'version':'1.0.0', 'nickName': 'Phone_Retail_Shop', 'description': 'Phone Sales', 'metaData':[ {'name':'clientType','type':'STRING'} ], 'payloadData':[ {'name':'brand','type':'STRING'}, {'name':'quantity','type':'INT'}, {'name':'total','type':'INT'}, {'name':'user','type':'STRING'} ]}
● Data Streams are versioned○ Allows for easier analysis
The Analyzer Engine
The Analyzer Engine● Powered by Apache Hadoop with querying/managing
through Apache Hive
● Data transfer to and from data sources is handled through custom storage handlers
● Parallel, distributed processing through the MapReduce programming model
• Runs on local Hadoop node or delegates to Hadoop cluster
• Scalable analytics
• Cluster can range from a couple of nodes to 1000s
● Analysis is carried out based on analytics scripts
● Scripts are based on an easy-to-learn, SQL-like query language
INSERT OVERWRITE TABLE UserTable SELECT userName, COUNT(DISTINCT orderID),SUM(quantity) FROM PhoneSalesTable WHERE version= "1.0.0" GROUP BY userName;
The Analyzer Engine
The Analyzer Engine
● Scripts can be scheduled○ Ex: once a minute, every Wednesday at 4:15 p.m., every
30th at 12 midnight
● Polyglot data architectures are supported○ Write summarised information to any kind of datastore
○ Custom written Hive JDBC handler used by default for output to relational DBs
The Presentation Layer
The Presentation Layer
• Gadget Portal for the dashboard
• Gadget Generation Wizard – Tool for generating custom
gadgets for the dashboard
• Activity Dashboard for correlating activities
• Message Console
• Plug in your own report server/ dashboard server
The Presentation LayerBAM Dashboards
The Presentation LayerGadget Portal
The Presentation Layer
Activity Dashboard
Activity Dashboard (cont'd)
The Presentation LayerMessage Console (HL7)
Message Console (cont'd)
BAM Toolboxes● BAM Toolboxes are installable and hot deployable artifacts used for
deploying functionalities to a BAM server
○ Stream definitions○ Analytics scripts○ Dashboards
● Supports plugging in of one or more of the above features
● Toolboxes for monitoring and auditing most WSO2 products are available OOTB
● Toolboxes for custom scenarios can be created easily
HA Distributed BAM deployment● WSO2 BAM can be clustered and deployed in a distributed manner to enable
high-availability, fail-over scenarios
○ Partially distributed deployment
■ Cassandra (storage) nodes are clustered
○ Fully distributed deployment
■ All components of BAM are clustered (data receiver, storage, analyzer and presentation)
● Hazelcast in-memory data grids are used for clustering implementation
Partially Distributed BAM deployment
Fully Distributed BAM deployment
Deployment – Shared events with CEP
Performance of WSO2 BAM
3-Node Cassandra Cluster used
Demo: From Publishing to Visualization
Monitoring a business process - Sales order handling
Verify order
Select supplier
Produce items
Ship products
In_production = true
Monitoring a business process - Business events
Verify order
Select supplier
Produce items
Ship products
In_production = true
Case ID = case_1Activity = Verify orderAssignee = SmithDuration = 20 mins
Case ID = case_1Activity = Verify orderAssignee = SmithDuration = 20 mins
Business events...
Case ID Activity Assignee Duration
Case_1 Verify order Smith 10
Case_2 Verify order John 15
Case_1 Produce items Nick 30
Case_1 Ship products Peter 150
Case_2 Produce items John 200
Case_3 Verify order Smith 20
...
WSO2 BAM - Deployment
Workflowsystem
BAM data publisher
Events store(e.g. Cassendra)
WSO2 BAM
Analyticsscripts
Results store(e.g. MySQL)
Gadget server
Gadgets
Visualizations /Reports
Demo: Activity Monitoring
Questions?
Thank You