FIWARE CEP GE introduction, ICT 2015

CEP GEInna Skarbovsky

Outline

1. Introduction to CEP

2. Working with the CEP GE

3. Available Documentation

4. Example

3

CEP Runtime

CEP

4

CEP Technology – expanding the ECA paradigm

From Event-Condition-Action to Pattern-Condition-Action In certain scenarios, single events are insignificant, a CEP engine can detect

combinations of events, and generate derived events, called situations, which are meaningful

CEPAuthoring

Tool

CEPEngine

Definitions

DetectedSituations

EventSources

Run Time

Build Time

Input Events

Event Processing Rules / Patterns

Detected Situations

5

CEP Technology – expanding the ECA paradigm

CEPAuthoring

Tool

CEPEngine

Definitions

DetectedSituations

EventSources

Run Time

Build Time

From Event-Condition-Action to Pattern-Condition-Action Use CEP when there is a need to detect patterns over incoming events Other tools can be used when there is a need to respond to single events

with some condition (e.g., Context Broker GE)

Input Events

Event Processing Rules / Patterns

Detected Situations

The Event Processing Network

An event processing network (EPN) describes the event processing flow execution. It consists of a collection of event processing agents (EPAs), producers, and consumers linked by channels.

Event Processing Agent (EPA) - Pattern Examples

Basic type: A lamp battery charge is below 15 percent

Aggregate type:The percentage of failed measurements is higher than 10 percent in the last 5 minutes

Absent type: no sensor events arrived in the last 10 minutes

Event Processing Agent (EPA) - Pattern Examples

Sequence type: The same customer bought and later sold the same stock within 24 hours

All type: Alert if customer performs a withdrawal and a deposit, irrelevant in which order, within 10 mins

Processing Context Definition

Every EPA (event processing agent) is associated with a processing context.

A processing context groups event instances so that they can be processed in a related way.

Processing Context Examples

Temporal Context

“5 seconds after sensor event with high value”

“A time window bounded by Order-Placed and Order-Delivered events”

“From each sensor event for 10 mins”

“Sensor’s ID”“Shipment’s ID”“Building’s ID”

Segmentation Context

“All events of the same “Sensor’s ID” arrived in the last hour”“All sensor events that arrived 5 seconds after sensor event with high value, all arriving from the same building”

Composite context – group of several contexts +

2. Working with the Proton (Proactive Technology Online) CEP GE Instance

12

Build Time Web User Interface

Events Context

Agents

Validation Layer

JSON Object

13

Proton (CEP GE Instance) – Build Time Web User Interface

Definition types:• Event types• EPAs (Event Processing Agents) - From various types: Basic,

All, Sequence, Absent, Aggregation• Contexts (Temporal, Segmentation, Composite)• Producers – From various types: File, REST, Custom• Consumers – From various types: File, REST, Custom

14

Creating Definitions

• Can be generated through the WEB User Interface

• Can be generated by external systems– In the expected JSON format

15

Runtime Main Interactions

ProtonRuntime

Input

Events

Input Adapters

Files

REST Client

Custom

Output Events

ProtonRuntimeProtonRuntime

Rest Admin API

State

Output Adapters

Files

REST Client

Custom

Authoring Tool Web UIRest API

Push EventsREST Service

Engine StateDefinitions Repository

http://forge.fi-ware.org/plugins/mediawiki/wiki/fiware/index.php/Complex_Event_Processing_Open_RESTful_API_Specification

16

Runtime Main Interactions (1) – Input & Output Adapters

ProtonRuntime

Input

Events

Input Adapters

Files

REST Client

Custom

Output Events

ProtonRuntimeProtonRuntime

Output Adapters

Files

REST Client

Custom

Producers Consumers

17

Proton (CEP GE Instance) – Rest Input Adapter

• Rest Input Adapter Client– Has a Producer definition in the Web UI

• Activates a REST service as a client – Access the REST web service declared by the event producer

and pull events using the GET method (pull approach)

Supported formats:– Tag delimited:Name=ShipPosition;ShipID=RTX33;Long=46;Lat=55;Speed=4.0;Time=13330330;

– Json:{"Name":"TrafficReport", "volume":"1000"}

18

Proton (CEP GE Instance) – Rest Output Adapter• Rest Output Adapter Client

– Has a Consumer definition in the Web UI• Activates a REST service as a client

– Access the REST web service declared by the event consumer and push events to the consumer using the POST method

Supported formats:– Tag delimited:Name=TrafficReport;Certainty=0.0;Cost=0.0;EventSource=;OccurrenceTime=null;

Annotation=;Duration=0.0;volume=1000; EventId=40f68052-3c7c-4245-ae5a-6e20def2e618;ExpirationTime=null;Chronon=null;DetectionTime=1349181899221;

– JSON (flat):{"Cost":"0.0","Certainty":"0.0","Name":"TrafficReport","EventSource":"","Duration":

"0.0","Annotation":"", "volume":"1000","EventId":"e206b5e8-9f3a-4711-9f46-d0e9431fe215","DetectionTime":"1350311378034"}

– NGSI XML – For sending output events to the Context Broker

19

Runtime Main Interactions (2) - Push events REST API

ProtonRuntime

Input

Events

Input Adapters

Files

REST Client

Custom

Output Events

ProtonRuntimeProtonRuntime

Output Adapters

Files

REST Client

Custom

Rest API

Push EventsREST Service

REST API for event push

POST http://{host}:{port}/ProtonOnWebServer/rest/events

Header: Content-Type: application/json

Data: {"Name":"TrafficReport", "volume":"1000"}

Supported event formats:•Tag delimited•JSON•NGSI-XML – allows to get events from the Context Broker

21

Runtime Interactions(3) – REST admin API

ProtonRuntimeProtonRuntimeProtonRuntime

Rest Admin API

State

Authoring Tool Web UI

Engine StateDefinitions Repository

REST Admin APIs

• Managing the Definitions Repository

– GET all the existing definitions in the repository – GET specific definition in JSON format– POST a new definition– PUT update for a definition– DELETE definition

REST Admin APIs

Administrating runtime Instances

GET instance status

PUT instance status

Status:– The instance’s definition URI– The instance’s state (stopped or started)

CEP GE Documentation• Start from the FI-WARE Catalogue

• Under the Complex Event Processing (CEP) - IBM Proactive Technology Online GE you can find links to:

– Open specification - Restful API– The CEP user guide and programmer guide– The CEP Installation and administration guide– CEP open source code– CEP image– CEP tutorial

• The Context Broker- CEP connection instructions can be found in the CEP User guide (Appendix – page 40 and on) - https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/CEP_GE_-_IBM_Proactive_Technology_Online_User_and_Programmer_Guide

Thank you!

Inna Skarbovsky([email protected])

BACKUP

Pattern: Expected CrashWhen there is a sequence of 3 TrafficReport events, with more than 50% volume increase between each two events,while all arrived within 3 minutes after startup,derive an Expected Crash output event

Example

3 minutes after startup window

TrafficReport

TrafficReport

TrafficReport

Expected Crash

ExampleEPA

Context

Example

• Experimental Authoring tool URL:http://130.206.81.23:8080/AuthoringTool/Main.html

Example Project: DoS AttackEvent: Traffic Report (input), Expected Crash(output)EPA: Increasing Traffic (see previous slide)Temporal Processing Context – 3 minutes After Startup Consumer: Report to file (DoSAttack_PredictedCrash.txt)

• Export definition to definition repository

Example

• Check engine instance state (Run status, Used definition file)

Expected Result: Status:200 OK

• Activate engine

Expected Result: Status:200 OK

GET http://{host}:{port}/ProtonOnWebServerAdmin/resources/instances/ProtonOnWebServer

{"state":"stopped ","definitions-url":"\/ProtonOnWebServerAdmin\/resources\/definitions\/DoSAttack2"}

PUT http://{host}:{port}/ProtonOnWebServerAdmin/resources/instances/ProtonOnWebServer{"action":"ChangeState","state":"start"}

Example• Post 3 input events

Expected Result: Status:200 OK

• Examine output file:– Tomact-Installation-dir/sample/DoSAttack_PredictedCrash.txt See

log file:– Tomact-Installation-dir/logs/catalina.out

POST http://{host}:{port}/ProtonOnWebServer/rest/eventsHeader: Content-Type: application/json

Data: {"Name":"TrafficReport", "volume":"1000"}Data: {"Name":"TrafficReport", "volume":"1600"}Data: {"Name":"TrafficReport", "volume":“2500"}