ESM 7 Distributed Correlation - Micro Focus · ESM 7 Distributed Correlation Paul MacGyver Carman Global Technical Security Sales Engineer. Confidential information This is a rolling

#MicroFocusCyberSummit

ESM 7 Distributed CorrelationPaul MacGyver Carman

Global Technical Security Sales Engineer

Confidential informationThis is a rolling (up to three year) Roadmap and is subject to change without notice

2

Confidential information. This Roadmap contains Confidential Information of Micro Focus and/or

its affiliates (“Micro Focus”), and is subject to change without notice. If you have a valid

Confidential Disclosure Agreement (“CDA”) with a Micro Focus entity, use of the Roadmap is

subject to that CDA and allowed solely for the purpose of evaluating purchase decisions from

Micro Focus. If not, it is subject to the following terms. For 3 years after disclosure, You may use

the Roadmap solely for the purpose of evaluating purchase decisions from Micro Focus. You

must use a reasonable standard of care to prevent disclosure. You will not disclose the contents

of the Roadmap to any third party with Micro Focus’ prior written approval unless it first

becomes publically known or is rightfully received by you from a third party without duty of

confidentiality.

Why Distributed Correlation?

Distributed Correlation Architecture

Components and Relationship

What gets processed where?

Deployment and Sizing Examples

Tips and Requirements

Monitoring the ESM Cluster

Next Steps

3

Agenda

Massively scales up to 100,000 correlated events per second per cluster

Easily add nodes to scale out

No more multi-tier configurations needed to scale out

No more forwarding connectors required between layers

Write more content, support heavier content

Why distributed correlation?

Split data stream to parallel processes across multiple nodes in a cluster

Clustered correlation

Centralized Alerts and Content building

HPE Confidential, under NDA use only

Architecture

The ESM 7 Distributed Correlation Architecture

6

Persistor

UI Interaction

ACCConsole Connectors / EB

Legend

Data Exchange

Event Intakeand Persistence

Repo InformationExchange

Isolate persistence

CORRE DB


7

Persistor

UI Interaction


Legend

Data Exchange



CorrelatorsCorrelators

Correlators: filter evaluation

CORRE DB


8

UI Interaction

Legend

Data Exchange



AggregatorsAggregators

Aggregators: grouping

Persistor



CORRE DB


Persistor



CORRE DB


9

UI Interaction

Legend

Data Exchange



Message Bus: events, data

Message

Bus


Persistor



CORRE DB

Message

Bus


10

UI Interaction

Legend

Data Exchange



Distributed Cache: lists, resources

Distributed

Cache


Persistor



CORRE DB

Message

Bus

Distributed

Cache


11

UI Interaction

Legend

Data Exchange



Repository: global settings

Repository

Event Flow … In a Nutshell

12

AggregatorsCorrelatorsPersistor AggregatorsCorrelatorsConnectors,

EB, ESM, ...

Persisted & Enriched

Correlation,Audit

Rule

DataMonitor

Audit Correlation,Audit

Audit

Where correlation happens

Pre-persistence Rules Light-weight Rules Standard Rules,Data Monitors

Deployment and Sizing

14

Sample 3 Node Configuration

Persistor

Mbus_control

Repo

Correlator

Aggregator

Mbus_control

Mbus_data

Repo

Correlator

Aggregator

Mbus_control

Mbus_data

Repo

ESM Node 1 ESM Node 2 ESM Node 3

Includes DCache)

Persistor deployed to single node

Number of correlator, aggregator, mbus, distributed cache and repository can be adjusted as needed across nodes

Actual layout of services may be changed based on capacity requirements

2 Correlators are recommended if the number of cores is 24 or greater, and the network is 10 Gbit or greater.

15

Sample 4 Node Configuration - PREFERRED

ESM Node 1 ESM Node 2 ESM Node 4ESM Node 3

Persistor

Repo

Correlator

Aggregator

DCache

Mbus_data

Mbus_control

Correlator

Correlator

Repo

Mbus_data

Mbus_control

Aggregator

Aggregator

DCache

Mbus_data

Mbus_control

RepoIncludes DCache)

Message Bus control deployed to three nodes due to need for odd number deployment requirement


Number of correlator, aggregator, distributed cache and repository can be adjusted as needed across nodes


1 Correlator and 2 Aggregators are recommended if the number of cores is 32 or greater

16

Sample 5 Node Configuration

ESM Node 1

Persistor

Repo

ESM Node 2

Correlator

Correlator

Aggregator

DCache

Mbus_cont

Mbus_data

ESM Node 3

Correlator

Correlator

Aggregator

Repo

Mbus_cont

Mbus_data

ESM Node 4

Correlator

Correlator

Aggregator

DCache

Mbus_cont

Mbus_data

Repo

ESM Node 5

DCache

Mbus_data

Includes DCache)

Correlator

Correlator

Aggregator

Message Bus control deployed to three nodes due to need for odd number deployment requirement


Number of correlator, aggregator, distributed cache and repository can be adjusted as needed across nodes


Isolate persistor as much as possible

Do not put mbus data on persistor node

3 or 5 mbus data nodes is best for redundancy

Correlators and Aggregators make a good pair for a node, usually 2C:1A

Persistor includes embedded DCache

Aggregators and Mbus_Data are very memory intensive – no more than 3 total on a node

Multiple Repos for redundancy is a good idea

Cluster supports one persistorinstance

17

Deployment Tips

No need for hardware homogeneity

Must be same network protocol

No direct connection required

Must be in the same data center

Same time zone for all nodes

Same OS and same OS version

18

Cluster Requirements

7.0 Cluster Monitoring Features

Dashboard in ACC to monitor cluster

“check engine light” in the console

Manage connectivity to MB and DC

Manage lags for correlator and aggregators

Manage.jsp is updated

20

Monitoring the ESM 7 cluster

Next Steps

Next Steps

Download ESM 7 Available to all customers

under maintenance

Can be deployed in Compact andDistributed Modes

Download and Reviewthe Documentation

Available from Protect 724

Review your Requirementswith SE and/or PS

Find out if distributed correlationis right for you

Thank You.



ESM 7 Distributed Correlation - Micro Focus · ESM 7 Distributed Correlation Paul MacGyver Carman Global Technical Security Sales Engineer. Confidential information This is a rolling

Documents