Real- Time Analytics – Process Automation

This document is offered compliments of BSP Media Group. www.bspmediagroup.com

All rights reserved.

First National Bank – a division of FirstRand Bank Limited. An Authorised Financial Services and Credit Provider (NCRCP20).

Real-Time Analytics Process Automation

Nico Coetzee

[email protected]

Big Data

• Volume

• Amount of data

• Velocity

• speed of data in and out

• Variety

• Range of data types and sources

Volume

• Examples:

• User profile database

• Inventory

• Real Time Analytics Scenarios:

• Measure changes over time

• Assist ERP systems with automation (order

stock)

• Stock load balancing (automatically redistribute

stock from one area to another)

Velocity

• Examples:

• POS Transactions (think big national retailers)

• Logs (firewall logs)

• Real Time Analytics Scenarios

• Sudden unexpected patterns (a region

experiences an outage)

• Attacks and other anomalies that can be picked

up from logs

Variety

• Examples:

• End-to-end transaction flows through Web

logs, Application server logs and database logs

• Real Time Analytics Scenarios:

• Continues monitor response times (very handy

for Cloud type solution where decisions to start

more VM’s may be required)

• Context required for making intelligent

decisions, for example in anti-fraud systems

The “Missing” V’s

• Viability

• The secret is uncovering the latent, hidden

relationships among these variables.

• Value

• Remember: Correlation does not mean

causation

• Realistic scenarios: Was your marketing

campaign successful?

Technologies to Consider

• MongoDB

• NoSQL DB

• Distributed operations (Grid FS)

• Built-in Map-Reduce engine

• Syslog-ng

• Real time decisions on log events

• Granular control over log routing

• Rules based on regular expressions

The Future

• DevOps and Agile methodologies can

benefit from the inputs from real time Big

Data analytics

• Identified (potential) defects should naturally

flow back into the backlog.

• Infrastructure and resource management

Thank You