BeepBeep 3: A declarative event stream query engine (EDOC 2015)

A declarative event streamquery engine

Booleans

B

Numbers

R

234

π

Strings

S

abc

Functions

X Y→

Sets

X2

A processor is a function that takes 0 or moreevent traces as input, and returns 0 or 1event trace as output

. . . . . .

Processors can be composed: the output ofa processor can be given as the input ofanother ("piping")

BeepBeep is an event stream query engine that provides...

A set of basic processors (independent fromany event type)

A core grammar and interpreter tocompose ("pipe") processors

Mechanisms to extend the grammarwith user-defined events, processors, etc.

abc. . . a . . .n cΨ

abc. . . b . . .n c

EVERY nTH OF (T)

TRIM n FROM (T)

WHERE2 1 5 0 2 0

C

C2 1 5 0

(T) WHERE condition

COMBINE (T) WITH f

Σ+2 1 5 0 2 3 8

f

f

8

2 1 5 0 2 1 5 0

f1 5 0 1 5 0

f5 0 5 0

++

+

a . . .b

a . . .b

FILE "filename"

SAVE (T) TO "filename"

WHEN @P IS A PROCESSOR:

THE SUM OF ( @P )

IS THE PROCESSOR

COMBINE (@P) WITH SUM.

Arbitrarysymbol

Grammar rule this symbolmust parse against

New grammar case

Grammar rule the caseis added toExpression the new casestands for

WHEN @P IS A PROCESSOR: THE COUNT OF ( @P ) IS THE PROCESSOR COMBINE (SELECT 1 FROM (@P)) WITH SUM.

WHEN @P IS A PROCESSOR: THE SUM OF ( @P ) IS THE PROCESSOR COMBINE (@P) WITH SUM.

WHEN @P IS A PROCESSOR: THE AVERAGE OF ( @P ) IS THE PROCESSOR SELECT (T.*) ÷ (U.*) FROM ( THE SUM OF (@P) AS T, THE COUNT OF (@P) AS U).

On every fifth trading day starting today,calculate the average closing price of MSFT for the five most recent trading days, and keepthe query standing for fifty trading days.

timestamp stockSymbol closingPrice

0 APPL 1039.30 MSFT 950.00 GOGL 433.31 MSFT 951.21 APPL 1038.3... ... ...


String line = br.readLine().trim();if (!line.isEmpty()) { String[] parts = line.split(","); if (parts[0].compareTo("ABC") != 0) { value_index++; sum += Double.parseDouble(parts[1]); if (value_index == 5) { double average = sum / 5; value_index = 0; sum = 0; return average;}}}


SELECT afd FROM ( SELECT S1.timestamp AS ts, AVG(S2.closingPrice) AS afd FROM (SELECT * FROM stocks WHERE stockSymbol = "MSFT") AS S1, (SELECT * FROM stocks WHERE stockSymbol = "MSFT") AS S2 WHERE (S2.timestamp - S1.timestamp) < 5 GROUP BY S1.timestamp) AS S3WHERE MOD(ts, 5) = 0;


EVERY 5TH OF ( APPLY (THE AVERAGE OF (*)) TO ( SELECT closingPrice FROM stocks) WHERE (stockSymbol) = ("MSFT")))) ON A WINDOW OF 5).

Calculate how many times the closing priceof MSFT is greater than 20 and the nextday, its closing price is less than 10.


SELECT COUNT(*) FROM (SELECT * FROM stocks WHERE stockSymbol = "MSFT") AS S1, (SELECT * FROM stocks WHERE stockSymbol = "MSFT") AS S2 WHERE (S2.timestamp - S1.timestamp) = 1 AND S1.closingPrice > 20 AND S2.closingPrice < 10;


WHEN @P IS A PROCESSOR: MY PATTERN IN ( @P ) IS THE PROCESSOR (SELECT (closingPrice) LESS THAN (20) FROM (@P)) AND (NEXT (SELECT (closingPrice) GREATER THAN (10) FROM (@P))).

THE COUNT OF (( MY PATTERN IN ( (SELECT closingPrice FROM stocks) WHERE (stockSymbol) = ("MSFT"))) WHERE (*) = (true)).

LinearTemporalLogic!

import ca.uqac.lif.cep.*;

public class MyProcessor extends SingleProcessor {

public Queue<Vector<Object>> compute(Vector<Object> inputs) {

}

public void build(Stack<Object> s) {

}}

. . . Create output events from input . . .

. . . Instantiate processor from parse stack . . .

<processor> := . . .<number> := . . .<string> := . . .

Add new rules to any symbol fromthe basic grammar

<processor> := <my_processor> ;

<my_processor> := INCREMENT ( <processor> ) BY <number> ;

Symbols already defined in basic grammar

Adds a new case to an existing rule

import ca.uqac.lif.cep.*;

public class MyProcessor extends SingleProcessor {

private int increment;

public Queue<Vector<Object>> compute(Vector<Object> inputs) { Queue<Vector<Object>> out = new Queue<Vector<Object>>(); Vector<Object> v = new Vector<Object>(); Integer i = (Integer) inputs.firstElement() + increment; v.addElement(i); out.put(v); return out; }

. . .

. . .

public void build(Stack<Object> s) { Number n = (Number) s.pop(); s.pop(); s.pop(); Processor p = (Processor) s.pop(); s.pop(); s.pop();

increment = n.intValue();

Connector.connect(p, this);

s.push(this); }}

Read contents ofparse stack

<number>BY(<processor>)INCREMENT

Set processor's statePipe it to its inputPut on parse stack

Some pre-packaged grammar extensions:

Manipulation of name-value tuples

Set theory

Formatted input (CSV, XML, JSON)

Graphing (histograms, scatterplots, ...)

Basic signal processing (smoothing, peak detection, ...)

Create your own!

https://bitbucket.org/sylvainhalle/beepbeep-3

import ca.uqac.lif.cep.*;import ca.uqac.lif.cep.eml.tuples.*;

public class MyExample {

public static void main(String[] args) { Interpreter my_int = new Interpreter();

my_int.extendGrammar(TupleGrammar.class);

Pullable p = my_int.executeQuery( "\"HELLO WORLD\"");

for (int i = 0; i < 10; i++) { EmlString s = (EmlString) p.pull(); System.out.println(s); } }}

Create query interpreterLoad a grammar extension

Execute a query

Pull an output event

BeepBeep 3: A declarative event stream query engine (EDOC 2015)

Technology