XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Post on 18-Dec-2015

231 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

Transcript

XFA Faster Signature Matching With Extended AutomataAuthor Randy Smith Cristian Estan and Somesh Jha

Publisher IEEE Symposium on Security and Privacy 2008

Presenter Yu-Hao Tseng

Date 20140115

1

Outline

bull Introductionbull Technical overviewbull Build XFAs from Regexbull Experimental Results

2

Introduction

bull In this paper their primary goal is to improve the time and space efficiency of signature matching in network intrusion detection systems (NIDS)

bull To achieve their goal they introduce extended finite automata (XFAs) which augment traditional FSAs with a finite scratch memory used to remember various types of information relevant to the progress of signature matching

3

Technical overview

bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

4

Technical overview (Cont)

bull where all and are distinct strings

5

Technical overview (Cont)

bull where all and are distinct stringsbull use a single bit of scratch memory

6

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion

    Outline

    bull Introductionbull Technical overviewbull Build XFAs from Regexbull Experimental Results

    2

    Introduction

    bull In this paper their primary goal is to improve the time and space efficiency of signature matching in network intrusion detection systems (NIDS)

    bull To achieve their goal they introduce extended finite automata (XFAs) which augment traditional FSAs with a finite scratch memory used to remember various types of information relevant to the progress of signature matching

    3

    Technical overview

    bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

    bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

    4

    Technical overview (Cont)

    bull where all and are distinct strings

    5

    Technical overview (Cont)

    bull where all and are distinct stringsbull use a single bit of scratch memory

    6

    Technical overview (Cont)bull which consists of all strings of length n

    bull use a counter

    7

    Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

    bull Changes shape of automatabull Tames state space explosion

    8

    Build XFAs from Regex

    bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

    bull ε ndasheliminationbull determinizing transitionsbull data determinization

    bull Finding efficient implementations

    9

    Build XFAs from Regex (Cont)

    bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

    bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

    bull ex abcd =gt abcd

    10

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Definitions

    bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

    data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

    an initial data value d0bull F QtimesD is the set of accepting configurationssube

    11

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Definitions

    bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

    transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

    defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

    12

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull From parse trees to NXFAs

    13

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull From parse trees to NXFAs

    bull Ex ab[^a]1 =gt ab[^a]1

    14

    ａ

    ｂ

    sum

    1

    [ a]

    1

    2

    0

    3

    4

    bit = 0cnt = 0 sum

    a

    bsum

    [^a]

    ε

    ε

    ε

    cnt++

    if (bit == 1 ampamp cnt = 1) accept()

    bit = 1

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull From parse trees to NXFAs

    bull Ex ab[^a]1 =gt ab[^a]1

    15

    1

    2

    0

    3

    4

    bit = 0cnt = 0 sum

    a

    bsum

    [^a]

    ε

    ε

    ε

    cnt++

    if (bit == 1 ampamp cnt = 1) accept()

    bit = 1

    120576

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

    16

    120576

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

    17

    120576

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

    18

    120576

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

    Qrsquo A C

    Drsquo (A 0) (B 1) (C 0) (C 2)

    δlsquo

    A C

    Frsquo

    19

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

    Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

    Drsquo (A 0) (B 1) (C 0) (C 2)

    δlsquo A C A C

    A C

    Frsquo

    A C

    A C

    20

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

    δlsquo A C A C A C A B C

    A C

    Frsquo

    A B C

    A C

    21

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

    δlsquo A C A C A C A B C A B C A B C

    A C

    Frsquo

    A B C

    A B C

    22

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

    Drsquo (A 0) (B 0) (C 0) (C 2)

    δlsquo A C A C A C A B C A B C A B C A B C A C

    A C

    Frsquo

    A C

    A B C

    23

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

    Drsquo (A 0) (B 0) (C 0) (C 2)

    δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

    A C

    Frsquo

    A C

    A B C

    24

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

    Drsquo (A 0) (B 0) (C 0) (C 2)

    δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

    A C

    Frsquo (A C (C 1)) (A B C (C 1))

    A C

    A B C

    25

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H

    Drsquo 3 5

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5)

    26

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5

    Drsquo 3 5

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5)

    G G

    35 35

    27

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5)

    G H

    35 3 4 5

    28

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5)

    H H

    3 4 5 3 4 5

    29

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

    Drsquo 3 5 3 4 5 3 5 7

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5) (G 3 5 7)

    H G

    3 4 5 3 5 7

    30

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

    Drsquo 3 5 3 4 5 3 5 7

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5) (G 3 5 7)

    H G

    3 4 5 3 5

    31

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

    Drsquo 3 5 3 4 5 3 5 7 3 5 6

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

    G G

    3 5 7 3 5 6

    32

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

    Drsquo 3 5 3 4 5 3 5 7 3 5 6

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

    G H

    3 5 7 3 4 5

    33

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

    Drsquo 3 5 3 4 5 3 5 7 3 5 6

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

    G G

    3 5 6 3 5

    34

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

    Drsquo 3 5 3 4 5 3 5 7 3 5 6

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo

    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

    G H

    3 5 6 3 4 5

    35

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Alg 3 data determinization for NXFAs

    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

    Drsquo 3 5 3 4 5 3 5 7 3 5 6

    δlsquo G G G H H H H G H G

    ( ) (G 3 5)

    Frsquo (G 3 5 6)

    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

    G H

    3 5 6 3 4 5

    36

    Build XFAs from Regex (Cont)

    bull Compile to XFAbull Finding efficient implementations

    37

    Experimental Results

    bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

    38

    Experimental Results (Cont)

    39

    Experimental Results (Cont)

    40

    Conclusion

    bull DFAs for regular expressions often blow up when combined

    bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

    bull Result compared to other feasible approaches reduce both time and space

    41

    • XFA Faster Signature Matching With Extended Automata
    • Outline
    • Introduction
    • Technical overview
    • Technical overview (Cont)
    • Technical overview (Cont) (2)
    • Technical overview (Cont) (3)
    • Technical overview (Cont) (4)
    • Build XFAs from Regex
    • Build XFAs from Regex (Cont)
    • Build XFAs from Regex (Cont) (2)
    • Build XFAs from Regex (Cont) (3)
    • Build XFAs from Regex (Cont) (4)
    • Build XFAs from Regex (Cont) (5)
    • Build XFAs from Regex (Cont) (6)
    • Build XFAs from Regex (Cont) (7)
    • Build XFAs from Regex (Cont) (8)
    • Build XFAs from Regex (Cont) (9)
    • Build XFAs from Regex (Cont) (10)
    • Build XFAs from Regex (Cont) (11)
    • Build XFAs from Regex (Cont) (12)
    • Build XFAs from Regex (Cont) (13)
    • Build XFAs from Regex (Cont) (14)
    • Build XFAs from Regex (Cont) (15)
    • Build XFAs from Regex (Cont) (16)
    • Build XFAs from Regex (Cont) (17)
    • Build XFAs from Regex (Cont) (18)
    • Build XFAs from Regex (Cont) (19)
    • Build XFAs from Regex (Cont) (20)
    • Build XFAs from Regex (Cont) (21)
    • Build XFAs from Regex (Cont) (22)
    • Build XFAs from Regex (Cont) (23)
    • Build XFAs from Regex (Cont) (24)
    • Build XFAs from Regex (Cont) (25)
    • Build XFAs from Regex (Cont) (26)
    • Build XFAs from Regex (Cont) (27)
    • Build XFAs from Regex (Cont) (28)
    • Experimental Results
    • Experimental Results (Cont)
    • Experimental Results (Cont) (2)
    • Conclusion

      Introduction

      bull In this paper their primary goal is to improve the time and space efficiency of signature matching in network intrusion detection systems (NIDS)

      bull To achieve their goal they introduce extended finite automata (XFAs) which augment traditional FSAs with a finite scratch memory used to remember various types of information relevant to the progress of signature matching

      3

      Technical overview

      bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

      bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

      4

      Technical overview (Cont)

      bull where all and are distinct strings

      5

      Technical overview (Cont)

      bull where all and are distinct stringsbull use a single bit of scratch memory

      6

      Technical overview (Cont)bull which consists of all strings of length n

      bull use a counter

      7

      Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

      bull Changes shape of automatabull Tames state space explosion

      8

      Build XFAs from Regex

      bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

      bull ε ndasheliminationbull determinizing transitionsbull data determinization

      bull Finding efficient implementations

      9

      Build XFAs from Regex (Cont)

      bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

      bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

      bull ex abcd =gt abcd

      10

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Definitions

      bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

      data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

      an initial data value d0bull F QtimesD is the set of accepting configurationssube

      11

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Definitions

      bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

      transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

      defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

      12

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull From parse trees to NXFAs

      13

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull From parse trees to NXFAs

      bull Ex ab[^a]1 =gt ab[^a]1

      14

      ａ

      ｂ

      sum

      1

      [ a]

      1

      2

      0

      3

      4

      bit = 0cnt = 0 sum

      a

      bsum

      [^a]

      ε

      ε

      ε

      cnt++

      if (bit == 1 ampamp cnt = 1) accept()

      bit = 1

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull From parse trees to NXFAs

      bull Ex ab[^a]1 =gt ab[^a]1

      15

      1

      2

      0

      3

      4

      bit = 0cnt = 0 sum

      a

      bsum

      [^a]

      ε

      ε

      ε

      cnt++

      if (bit == 1 ampamp cnt = 1) accept()

      bit = 1

      120576

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

      16

      120576

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

      17

      120576

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

      18

      120576

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

      Qrsquo A C

      Drsquo (A 0) (B 1) (C 0) (C 2)

      δlsquo

      A C

      Frsquo

      19

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

      Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

      Drsquo (A 0) (B 1) (C 0) (C 2)

      δlsquo A C A C

      A C

      Frsquo

      A C

      A C

      20

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

      δlsquo A C A C A C A B C

      A C

      Frsquo

      A B C

      A C

      21

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

      δlsquo A C A C A C A B C A B C A B C

      A C

      Frsquo

      A B C

      A B C

      22

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

      Drsquo (A 0) (B 0) (C 0) (C 2)

      δlsquo A C A C A C A B C A B C A B C A B C A C

      A C

      Frsquo

      A C

      A B C

      23

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

      Drsquo (A 0) (B 0) (C 0) (C 2)

      δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

      A C

      Frsquo

      A C

      A B C

      24

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

      Drsquo (A 0) (B 0) (C 0) (C 2)

      δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

      A C

      Frsquo (A C (C 1)) (A B C (C 1))

      A C

      A B C

      25

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H

      Drsquo 3 5

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5)

      26

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5

      Drsquo 3 5

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5)

      G G

      35 35

      27

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5)

      G H

      35 3 4 5

      28

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5)

      H H

      3 4 5 3 4 5

      29

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

      Drsquo 3 5 3 4 5 3 5 7

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5) (G 3 5 7)

      H G

      3 4 5 3 5 7

      30

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

      Drsquo 3 5 3 4 5 3 5 7

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5) (G 3 5 7)

      H G

      3 4 5 3 5

      31

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

      Drsquo 3 5 3 4 5 3 5 7 3 5 6

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

      G G

      3 5 7 3 5 6

      32

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

      Drsquo 3 5 3 4 5 3 5 7 3 5 6

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

      G H

      3 5 7 3 4 5

      33

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

      Drsquo 3 5 3 4 5 3 5 7 3 5 6

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

      G G

      3 5 6 3 5

      34

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

      Drsquo 3 5 3 4 5 3 5 7 3 5 6

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo

      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

      G H

      3 5 6 3 4 5

      35

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Alg 3 data determinization for NXFAs

      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

      Drsquo 3 5 3 4 5 3 5 7 3 5 6

      δlsquo G G G H H H H G H G

      ( ) (G 3 5)

      Frsquo (G 3 5 6)

      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

      G H

      3 5 6 3 4 5

      36

      Build XFAs from Regex (Cont)

      bull Compile to XFAbull Finding efficient implementations

      37

      Experimental Results

      bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

      38

      Experimental Results (Cont)

      39

      Experimental Results (Cont)

      40

      Conclusion

      bull DFAs for regular expressions often blow up when combined

      bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

      bull Result compared to other feasible approaches reduce both time and space

      41

      • XFA Faster Signature Matching With Extended Automata
      • Outline
      • Introduction
      • Technical overview
      • Technical overview (Cont)
      • Technical overview (Cont) (2)
      • Technical overview (Cont) (3)
      • Technical overview (Cont) (4)
      • Build XFAs from Regex
      • Build XFAs from Regex (Cont)
      • Build XFAs from Regex (Cont) (2)
      • Build XFAs from Regex (Cont) (3)
      • Build XFAs from Regex (Cont) (4)
      • Build XFAs from Regex (Cont) (5)
      • Build XFAs from Regex (Cont) (6)
      • Build XFAs from Regex (Cont) (7)
      • Build XFAs from Regex (Cont) (8)
      • Build XFAs from Regex (Cont) (9)
      • Build XFAs from Regex (Cont) (10)
      • Build XFAs from Regex (Cont) (11)
      • Build XFAs from Regex (Cont) (12)
      • Build XFAs from Regex (Cont) (13)
      • Build XFAs from Regex (Cont) (14)
      • Build XFAs from Regex (Cont) (15)
      • Build XFAs from Regex (Cont) (16)
      • Build XFAs from Regex (Cont) (17)
      • Build XFAs from Regex (Cont) (18)
      • Build XFAs from Regex (Cont) (19)
      • Build XFAs from Regex (Cont) (20)
      • Build XFAs from Regex (Cont) (21)
      • Build XFAs from Regex (Cont) (22)
      • Build XFAs from Regex (Cont) (23)
      • Build XFAs from Regex (Cont) (24)
      • Build XFAs from Regex (Cont) (25)
      • Build XFAs from Regex (Cont) (26)
      • Build XFAs from Regex (Cont) (27)
      • Build XFAs from Regex (Cont) (28)
      • Experimental Results
      • Experimental Results (Cont)
      • Experimental Results (Cont) (2)
      • Conclusion

        Technical overview

        bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

        bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

        4

        Technical overview (Cont)

        bull where all and are distinct strings

        5

        Technical overview (Cont)

        bull where all and are distinct stringsbull use a single bit of scratch memory

        6

        Technical overview (Cont)bull which consists of all strings of length n

        bull use a counter

        7

        Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

        bull Changes shape of automatabull Tames state space explosion

        8

        Build XFAs from Regex

        bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

        bull ε ndasheliminationbull determinizing transitionsbull data determinization

        bull Finding efficient implementations

        9

        Build XFAs from Regex (Cont)

        bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

        bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

        bull ex abcd =gt abcd

        10

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Definitions

        bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

        data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

        an initial data value d0bull F QtimesD is the set of accepting configurationssube

        11

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Definitions

        bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

        transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

        defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

        12

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull From parse trees to NXFAs

        13

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull From parse trees to NXFAs

        bull Ex ab[^a]1 =gt ab[^a]1

        14

        ａ

        ｂ

        sum

        1

        [ a]

        1

        2

        0

        3

        4

        bit = 0cnt = 0 sum

        a

        bsum

        [^a]

        ε

        ε

        ε

        cnt++

        if (bit == 1 ampamp cnt = 1) accept()

        bit = 1

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull From parse trees to NXFAs

        bull Ex ab[^a]1 =gt ab[^a]1

        15

        1

        2

        0

        3

        4

        bit = 0cnt = 0 sum

        a

        bsum

        [^a]

        ε

        ε

        ε

        cnt++

        if (bit == 1 ampamp cnt = 1) accept()

        bit = 1

        120576

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

        16

        120576

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

        17

        120576

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

        18

        120576

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

        Qrsquo A C

        Drsquo (A 0) (B 1) (C 0) (C 2)

        δlsquo

        A C

        Frsquo

        19

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

        Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

        Drsquo (A 0) (B 1) (C 0) (C 2)

        δlsquo A C A C

        A C

        Frsquo

        A C

        A C

        20

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

        δlsquo A C A C A C A B C

        A C

        Frsquo

        A B C

        A C

        21

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

        δlsquo A C A C A C A B C A B C A B C

        A C

        Frsquo

        A B C

        A B C

        22

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

        Drsquo (A 0) (B 0) (C 0) (C 2)

        δlsquo A C A C A C A B C A B C A B C A B C A C

        A C

        Frsquo

        A C

        A B C

        23

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

        Drsquo (A 0) (B 0) (C 0) (C 2)

        δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

        A C

        Frsquo

        A C

        A B C

        24

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

        Drsquo (A 0) (B 0) (C 0) (C 2)

        δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

        A C

        Frsquo (A C (C 1)) (A B C (C 1))

        A C

        A B C

        25

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H

        Drsquo 3 5

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5)

        26

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5

        Drsquo 3 5

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5)

        G G

        35 35

        27

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5)

        G H

        35 3 4 5

        28

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5)

        H H

        3 4 5 3 4 5

        29

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

        Drsquo 3 5 3 4 5 3 5 7

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5) (G 3 5 7)

        H G

        3 4 5 3 5 7

        30

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

        Drsquo 3 5 3 4 5 3 5 7

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5) (G 3 5 7)

        H G

        3 4 5 3 5

        31

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

        Drsquo 3 5 3 4 5 3 5 7 3 5 6

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

        G G

        3 5 7 3 5 6

        32

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

        Drsquo 3 5 3 4 5 3 5 7 3 5 6

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

        G H

        3 5 7 3 4 5

        33

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

        Drsquo 3 5 3 4 5 3 5 7 3 5 6

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

        G G

        3 5 6 3 5

        34

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

        Drsquo 3 5 3 4 5 3 5 7 3 5 6

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo

        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

        G H

        3 5 6 3 4 5

        35

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Alg 3 data determinization for NXFAs

        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

        Drsquo 3 5 3 4 5 3 5 7 3 5 6

        δlsquo G G G H H H H G H G

        ( ) (G 3 5)

        Frsquo (G 3 5 6)

        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

        G H

        3 5 6 3 4 5

        36

        Build XFAs from Regex (Cont)

        bull Compile to XFAbull Finding efficient implementations

        37

        Experimental Results

        bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

        38

        Experimental Results (Cont)

        39

        Experimental Results (Cont)

        40

        Conclusion

        bull DFAs for regular expressions often blow up when combined

        bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

        bull Result compared to other feasible approaches reduce both time and space

        41

        • XFA Faster Signature Matching With Extended Automata
        • Outline
        • Introduction
        • Technical overview
        • Technical overview (Cont)
        • Technical overview (Cont) (2)
        • Technical overview (Cont) (3)
        • Technical overview (Cont) (4)
        • Build XFAs from Regex
        • Build XFAs from Regex (Cont)
        • Build XFAs from Regex (Cont) (2)
        • Build XFAs from Regex (Cont) (3)
        • Build XFAs from Regex (Cont) (4)
        • Build XFAs from Regex (Cont) (5)
        • Build XFAs from Regex (Cont) (6)
        • Build XFAs from Regex (Cont) (7)
        • Build XFAs from Regex (Cont) (8)
        • Build XFAs from Regex (Cont) (9)
        • Build XFAs from Regex (Cont) (10)
        • Build XFAs from Regex (Cont) (11)
        • Build XFAs from Regex (Cont) (12)
        • Build XFAs from Regex (Cont) (13)
        • Build XFAs from Regex (Cont) (14)
        • Build XFAs from Regex (Cont) (15)
        • Build XFAs from Regex (Cont) (16)
        • Build XFAs from Regex (Cont) (17)
        • Build XFAs from Regex (Cont) (18)
        • Build XFAs from Regex (Cont) (19)
        • Build XFAs from Regex (Cont) (20)
        • Build XFAs from Regex (Cont) (21)
        • Build XFAs from Regex (Cont) (22)
        • Build XFAs from Regex (Cont) (23)
        • Build XFAs from Regex (Cont) (24)
        • Build XFAs from Regex (Cont) (25)
        • Build XFAs from Regex (Cont) (26)
        • Build XFAs from Regex (Cont) (27)
        • Build XFAs from Regex (Cont) (28)
        • Experimental Results
        • Experimental Results (Cont)
        • Experimental Results (Cont) (2)
        • Conclusion

          Technical overview (Cont)

          bull where all and are distinct strings

          5

          Technical overview (Cont)

          bull where all and are distinct stringsbull use a single bit of scratch memory

          6

          Technical overview (Cont)bull which consists of all strings of length n

          bull use a counter

          7

          Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

          bull Changes shape of automatabull Tames state space explosion

          8

          Build XFAs from Regex

          bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

          bull ε ndasheliminationbull determinizing transitionsbull data determinization

          bull Finding efficient implementations

          9

          Build XFAs from Regex (Cont)

          bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

          bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

          bull ex abcd =gt abcd

          10

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Definitions

          bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

          data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

          an initial data value d0bull F QtimesD is the set of accepting configurationssube

          11

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Definitions

          bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

          transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

          defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

          12

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull From parse trees to NXFAs

          13

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull From parse trees to NXFAs

          bull Ex ab[^a]1 =gt ab[^a]1

          14

          ａ

          ｂ

          sum

          1

          [ a]

          1

          2

          0

          3

          4

          bit = 0cnt = 0 sum

          a

          bsum

          [^a]

          ε

          ε

          ε

          cnt++

          if (bit == 1 ampamp cnt = 1) accept()

          bit = 1

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull From parse trees to NXFAs

          bull Ex ab[^a]1 =gt ab[^a]1

          15

          1

          2

          0

          3

          4

          bit = 0cnt = 0 sum

          a

          bsum

          [^a]

          ε

          ε

          ε

          cnt++

          if (bit == 1 ampamp cnt = 1) accept()

          bit = 1

          120576

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

          16

          120576

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

          17

          120576

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

          18

          120576

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

          Qrsquo A C

          Drsquo (A 0) (B 1) (C 0) (C 2)

          δlsquo

          A C

          Frsquo

          19

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

          Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

          Drsquo (A 0) (B 1) (C 0) (C 2)

          δlsquo A C A C

          A C

          Frsquo

          A C

          A C

          20

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

          δlsquo A C A C A C A B C

          A C

          Frsquo

          A B C

          A C

          21

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

          δlsquo A C A C A C A B C A B C A B C

          A C

          Frsquo

          A B C

          A B C

          22

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

          Drsquo (A 0) (B 0) (C 0) (C 2)

          δlsquo A C A C A C A B C A B C A B C A B C A C

          A C

          Frsquo

          A C

          A B C

          23

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

          Drsquo (A 0) (B 0) (C 0) (C 2)

          δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

          A C

          Frsquo

          A C

          A B C

          24

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

          Drsquo (A 0) (B 0) (C 0) (C 2)

          δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

          A C

          Frsquo (A C (C 1)) (A B C (C 1))

          A C

          A B C

          25

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H

          Drsquo 3 5

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5)

          26

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5

          Drsquo 3 5

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5)

          G G

          35 35

          27

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5)

          G H

          35 3 4 5

          28

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5)

          H H

          3 4 5 3 4 5

          29

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

          Drsquo 3 5 3 4 5 3 5 7

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5) (G 3 5 7)

          H G

          3 4 5 3 5 7

          30

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

          Drsquo 3 5 3 4 5 3 5 7

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5) (G 3 5 7)

          H G

          3 4 5 3 5

          31

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

          Drsquo 3 5 3 4 5 3 5 7 3 5 6

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

          G G

          3 5 7 3 5 6

          32

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

          Drsquo 3 5 3 4 5 3 5 7 3 5 6

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

          G H

          3 5 7 3 4 5

          33

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

          Drsquo 3 5 3 4 5 3 5 7 3 5 6

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

          G G

          3 5 6 3 5

          34

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

          Drsquo 3 5 3 4 5 3 5 7 3 5 6

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo

          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

          G H

          3 5 6 3 4 5

          35

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Alg 3 data determinization for NXFAs

          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

          Drsquo 3 5 3 4 5 3 5 7 3 5 6

          δlsquo G G G H H H H G H G

          ( ) (G 3 5)

          Frsquo (G 3 5 6)

          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

          G H

          3 5 6 3 4 5

          36

          Build XFAs from Regex (Cont)

          bull Compile to XFAbull Finding efficient implementations

          37

          Experimental Results

          bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

          38

          Experimental Results (Cont)

          39

          Experimental Results (Cont)

          40

          Conclusion

          bull DFAs for regular expressions often blow up when combined

          bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

          bull Result compared to other feasible approaches reduce both time and space

          41

          • XFA Faster Signature Matching With Extended Automata
          • Outline
          • Introduction
          • Technical overview
          • Technical overview (Cont)
          • Technical overview (Cont) (2)
          • Technical overview (Cont) (3)
          • Technical overview (Cont) (4)
          • Build XFAs from Regex
          • Build XFAs from Regex (Cont)
          • Build XFAs from Regex (Cont) (2)
          • Build XFAs from Regex (Cont) (3)
          • Build XFAs from Regex (Cont) (4)
          • Build XFAs from Regex (Cont) (5)
          • Build XFAs from Regex (Cont) (6)
          • Build XFAs from Regex (Cont) (7)
          • Build XFAs from Regex (Cont) (8)
          • Build XFAs from Regex (Cont) (9)
          • Build XFAs from Regex (Cont) (10)
          • Build XFAs from Regex (Cont) (11)
          • Build XFAs from Regex (Cont) (12)
          • Build XFAs from Regex (Cont) (13)
          • Build XFAs from Regex (Cont) (14)
          • Build XFAs from Regex (Cont) (15)
          • Build XFAs from Regex (Cont) (16)
          • Build XFAs from Regex (Cont) (17)
          • Build XFAs from Regex (Cont) (18)
          • Build XFAs from Regex (Cont) (19)
          • Build XFAs from Regex (Cont) (20)
          • Build XFAs from Regex (Cont) (21)
          • Build XFAs from Regex (Cont) (22)
          • Build XFAs from Regex (Cont) (23)
          • Build XFAs from Regex (Cont) (24)
          • Build XFAs from Regex (Cont) (25)
          • Build XFAs from Regex (Cont) (26)
          • Build XFAs from Regex (Cont) (27)
          • Build XFAs from Regex (Cont) (28)
          • Experimental Results
          • Experimental Results (Cont)
          • Experimental Results (Cont) (2)
          • Conclusion

            Technical overview (Cont)

            bull where all and are distinct stringsbull use a single bit of scratch memory

            6

            Technical overview (Cont)bull which consists of all strings of length n

            bull use a counter

            7

            Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

            bull Changes shape of automatabull Tames state space explosion

            8

            Build XFAs from Regex

            bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

            bull ε ndasheliminationbull determinizing transitionsbull data determinization

            bull Finding efficient implementations

            9

            Build XFAs from Regex (Cont)

            bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

            bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

            bull ex abcd =gt abcd

            10

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Definitions

            bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

            data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

            an initial data value d0bull F QtimesD is the set of accepting configurationssube

            11

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Definitions

            bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

            transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

            defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

            12

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull From parse trees to NXFAs

            13

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull From parse trees to NXFAs

            bull Ex ab[^a]1 =gt ab[^a]1

            14

            ａ

            ｂ

            sum

            1

            [ a]

            1

            2

            0

            3

            4

            bit = 0cnt = 0 sum

            a

            bsum

            [^a]

            ε

            ε

            ε

            cnt++

            if (bit == 1 ampamp cnt = 1) accept()

            bit = 1

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull From parse trees to NXFAs

            bull Ex ab[^a]1 =gt ab[^a]1

            15

            1

            2

            0

            3

            4

            bit = 0cnt = 0 sum

            a

            bsum

            [^a]

            ε

            ε

            ε

            cnt++

            if (bit == 1 ampamp cnt = 1) accept()

            bit = 1

            120576

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

            16

            120576

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

            17

            120576

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

            18

            120576

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

            Qrsquo A C

            Drsquo (A 0) (B 1) (C 0) (C 2)

            δlsquo

            A C

            Frsquo

            19

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

            Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

            Drsquo (A 0) (B 1) (C 0) (C 2)

            δlsquo A C A C

            A C

            Frsquo

            A C

            A C

            20

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

            δlsquo A C A C A C A B C

            A C

            Frsquo

            A B C

            A C

            21

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

            δlsquo A C A C A C A B C A B C A B C

            A C

            Frsquo

            A B C

            A B C

            22

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

            Drsquo (A 0) (B 0) (C 0) (C 2)

            δlsquo A C A C A C A B C A B C A B C A B C A C

            A C

            Frsquo

            A C

            A B C

            23

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

            Drsquo (A 0) (B 0) (C 0) (C 2)

            δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

            A C

            Frsquo

            A C

            A B C

            24

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

            Drsquo (A 0) (B 0) (C 0) (C 2)

            δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

            A C

            Frsquo (A C (C 1)) (A B C (C 1))

            A C

            A B C

            25

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H

            Drsquo 3 5

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5)

            26

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5

            Drsquo 3 5

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5)

            G G

            35 35

            27

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5)

            G H

            35 3 4 5

            28

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5)

            H H

            3 4 5 3 4 5

            29

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

            Drsquo 3 5 3 4 5 3 5 7

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5) (G 3 5 7)

            H G

            3 4 5 3 5 7

            30

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

            Drsquo 3 5 3 4 5 3 5 7

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5) (G 3 5 7)

            H G

            3 4 5 3 5

            31

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

            Drsquo 3 5 3 4 5 3 5 7 3 5 6

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

            G G

            3 5 7 3 5 6

            32

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

            Drsquo 3 5 3 4 5 3 5 7 3 5 6

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

            G H

            3 5 7 3 4 5

            33

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

            Drsquo 3 5 3 4 5 3 5 7 3 5 6

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

            G G

            3 5 6 3 5

            34

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

            Drsquo 3 5 3 4 5 3 5 7 3 5 6

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo

            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

            G H

            3 5 6 3 4 5

            35

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Alg 3 data determinization for NXFAs

            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

            Drsquo 3 5 3 4 5 3 5 7 3 5 6

            δlsquo G G G H H H H G H G

            ( ) (G 3 5)

            Frsquo (G 3 5 6)

            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

            G H

            3 5 6 3 4 5

            36

            Build XFAs from Regex (Cont)

            bull Compile to XFAbull Finding efficient implementations

            37

            Experimental Results

            bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

            38

            Experimental Results (Cont)

            39

            Experimental Results (Cont)

            40

            Conclusion

            bull DFAs for regular expressions often blow up when combined

            bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

            bull Result compared to other feasible approaches reduce both time and space

            41

            • XFA Faster Signature Matching With Extended Automata
            • Outline
            • Introduction
            • Technical overview
            • Technical overview (Cont)
            • Technical overview (Cont) (2)
            • Technical overview (Cont) (3)
            • Technical overview (Cont) (4)
            • Build XFAs from Regex
            • Build XFAs from Regex (Cont)
            • Build XFAs from Regex (Cont) (2)
            • Build XFAs from Regex (Cont) (3)
            • Build XFAs from Regex (Cont) (4)
            • Build XFAs from Regex (Cont) (5)
            • Build XFAs from Regex (Cont) (6)
            • Build XFAs from Regex (Cont) (7)
            • Build XFAs from Regex (Cont) (8)
            • Build XFAs from Regex (Cont) (9)
            • Build XFAs from Regex (Cont) (10)
            • Build XFAs from Regex (Cont) (11)
            • Build XFAs from Regex (Cont) (12)
            • Build XFAs from Regex (Cont) (13)
            • Build XFAs from Regex (Cont) (14)
            • Build XFAs from Regex (Cont) (15)
            • Build XFAs from Regex (Cont) (16)
            • Build XFAs from Regex (Cont) (17)
            • Build XFAs from Regex (Cont) (18)
            • Build XFAs from Regex (Cont) (19)
            • Build XFAs from Regex (Cont) (20)
            • Build XFAs from Regex (Cont) (21)
            • Build XFAs from Regex (Cont) (22)
            • Build XFAs from Regex (Cont) (23)
            • Build XFAs from Regex (Cont) (24)
            • Build XFAs from Regex (Cont) (25)
            • Build XFAs from Regex (Cont) (26)
            • Build XFAs from Regex (Cont) (27)
            • Build XFAs from Regex (Cont) (28)
            • Experimental Results
            • Experimental Results (Cont)
            • Experimental Results (Cont) (2)
            • Conclusion

              Technical overview (Cont)bull which consists of all strings of length n

              bull use a counter

              7

              Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

              bull Changes shape of automatabull Tames state space explosion

              8

              Build XFAs from Regex

              bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

              bull ε ndasheliminationbull determinizing transitionsbull data determinization

              bull Finding efficient implementations

              9

              Build XFAs from Regex (Cont)

              bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

              bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

              bull ex abcd =gt abcd

              10

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Definitions

              bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

              data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

              an initial data value d0bull F QtimesD is the set of accepting configurationssube

              11

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Definitions

              bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

              transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

              defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

              12

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull From parse trees to NXFAs

              13

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull From parse trees to NXFAs

              bull Ex ab[^a]1 =gt ab[^a]1

              14

              ａ

              ｂ

              sum

              1

              [ a]

              1

              2

              0

              3

              4

              bit = 0cnt = 0 sum

              a

              bsum

              [^a]

              ε

              ε

              ε

              cnt++

              if (bit == 1 ampamp cnt = 1) accept()

              bit = 1

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull From parse trees to NXFAs

              bull Ex ab[^a]1 =gt ab[^a]1

              15

              1

              2

              0

              3

              4

              bit = 0cnt = 0 sum

              a

              bsum

              [^a]

              ε

              ε

              ε

              cnt++

              if (bit == 1 ampamp cnt = 1) accept()

              bit = 1

              120576

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

              16

              120576

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

              17

              120576

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

              18

              120576

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

              Qrsquo A C

              Drsquo (A 0) (B 1) (C 0) (C 2)

              δlsquo

              A C

              Frsquo

              19

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

              Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

              Drsquo (A 0) (B 1) (C 0) (C 2)

              δlsquo A C A C

              A C

              Frsquo

              A C

              A C

              20

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

              δlsquo A C A C A C A B C

              A C

              Frsquo

              A B C

              A C

              21

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

              δlsquo A C A C A C A B C A B C A B C

              A C

              Frsquo

              A B C

              A B C

              22

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

              Drsquo (A 0) (B 0) (C 0) (C 2)

              δlsquo A C A C A C A B C A B C A B C A B C A C

              A C

              Frsquo

              A C

              A B C

              23

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

              Drsquo (A 0) (B 0) (C 0) (C 2)

              δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

              A C

              Frsquo

              A C

              A B C

              24

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

              Drsquo (A 0) (B 0) (C 0) (C 2)

              δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

              A C

              Frsquo (A C (C 1)) (A B C (C 1))

              A C

              A B C

              25

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H

              Drsquo 3 5

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5)

              26

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5

              Drsquo 3 5

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5)

              G G

              35 35

              27

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5)

              G H

              35 3 4 5

              28

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5)

              H H

              3 4 5 3 4 5

              29

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

              Drsquo 3 5 3 4 5 3 5 7

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5) (G 3 5 7)

              H G

              3 4 5 3 5 7

              30

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

              Drsquo 3 5 3 4 5 3 5 7

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5) (G 3 5 7)

              H G

              3 4 5 3 5

              31

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

              Drsquo 3 5 3 4 5 3 5 7 3 5 6

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

              G G

              3 5 7 3 5 6

              32

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

              Drsquo 3 5 3 4 5 3 5 7 3 5 6

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

              G H

              3 5 7 3 4 5

              33

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

              Drsquo 3 5 3 4 5 3 5 7 3 5 6

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

              G G

              3 5 6 3 5

              34

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

              Drsquo 3 5 3 4 5 3 5 7 3 5 6

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo

              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

              G H

              3 5 6 3 4 5

              35

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Alg 3 data determinization for NXFAs

              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

              Drsquo 3 5 3 4 5 3 5 7 3 5 6

              δlsquo G G G H H H H G H G

              ( ) (G 3 5)

              Frsquo (G 3 5 6)

              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

              G H

              3 5 6 3 4 5

              36

              Build XFAs from Regex (Cont)

              bull Compile to XFAbull Finding efficient implementations

              37

              Experimental Results

              bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

              38

              Experimental Results (Cont)

              39

              Experimental Results (Cont)

              40

              Conclusion

              bull DFAs for regular expressions often blow up when combined

              bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

              bull Result compared to other feasible approaches reduce both time and space

              41

              • XFA Faster Signature Matching With Extended Automata
              • Outline
              • Introduction
              • Technical overview
              • Technical overview (Cont)
              • Technical overview (Cont) (2)
              • Technical overview (Cont) (3)
              • Technical overview (Cont) (4)
              • Build XFAs from Regex
              • Build XFAs from Regex (Cont)
              • Build XFAs from Regex (Cont) (2)
              • Build XFAs from Regex (Cont) (3)
              • Build XFAs from Regex (Cont) (4)
              • Build XFAs from Regex (Cont) (5)
              • Build XFAs from Regex (Cont) (6)
              • Build XFAs from Regex (Cont) (7)
              • Build XFAs from Regex (Cont) (8)
              • Build XFAs from Regex (Cont) (9)
              • Build XFAs from Regex (Cont) (10)
              • Build XFAs from Regex (Cont) (11)
              • Build XFAs from Regex (Cont) (12)
              • Build XFAs from Regex (Cont) (13)
              • Build XFAs from Regex (Cont) (14)
              • Build XFAs from Regex (Cont) (15)
              • Build XFAs from Regex (Cont) (16)
              • Build XFAs from Regex (Cont) (17)
              • Build XFAs from Regex (Cont) (18)
              • Build XFAs from Regex (Cont) (19)
              • Build XFAs from Regex (Cont) (20)
              • Build XFAs from Regex (Cont) (21)
              • Build XFAs from Regex (Cont) (22)
              • Build XFAs from Regex (Cont) (23)
              • Build XFAs from Regex (Cont) (24)
              • Build XFAs from Regex (Cont) (25)
              • Build XFAs from Regex (Cont) (26)
              • Build XFAs from Regex (Cont) (27)
              • Build XFAs from Regex (Cont) (28)
              • Experimental Results
              • Experimental Results (Cont)
              • Experimental Results (Cont) (2)
              • Conclusion

                Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

                bull Changes shape of automatabull Tames state space explosion

                8

                Build XFAs from Regex

                bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

                bull ε ndasheliminationbull determinizing transitionsbull data determinization

                bull Finding efficient implementations

                9

                Build XFAs from Regex (Cont)

                bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

                bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

                bull ex abcd =gt abcd

                10

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Definitions

                bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

                data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

                an initial data value d0bull F QtimesD is the set of accepting configurationssube

                11

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Definitions

                bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

                transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

                defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

                12

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull From parse trees to NXFAs

                13

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull From parse trees to NXFAs

                bull Ex ab[^a]1 =gt ab[^a]1

                14

                ａ

                ｂ

                sum

                1

                [ a]

                1

                2

                0

                3

                4

                bit = 0cnt = 0 sum

                a

                bsum

                [^a]

                ε

                ε

                ε

                cnt++

                if (bit == 1 ampamp cnt = 1) accept()

                bit = 1

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull From parse trees to NXFAs

                bull Ex ab[^a]1 =gt ab[^a]1

                15

                1

                2

                0

                3

                4

                bit = 0cnt = 0 sum

                a

                bsum

                [^a]

                ε

                ε

                ε

                cnt++

                if (bit == 1 ampamp cnt = 1) accept()

                bit = 1

                120576

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                16

                120576

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                17

                120576

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                18

                120576

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                Qrsquo A C

                Drsquo (A 0) (B 1) (C 0) (C 2)

                δlsquo

                A C

                Frsquo

                19

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                Drsquo (A 0) (B 1) (C 0) (C 2)

                δlsquo A C A C

                A C

                Frsquo

                A C

                A C

                20

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                δlsquo A C A C A C A B C

                A C

                Frsquo

                A B C

                A C

                21

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                δlsquo A C A C A C A B C A B C A B C

                A C

                Frsquo

                A B C

                A B C

                22

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                Drsquo (A 0) (B 0) (C 0) (C 2)

                δlsquo A C A C A C A B C A B C A B C A B C A C

                A C

                Frsquo

                A C

                A B C

                23

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                Drsquo (A 0) (B 0) (C 0) (C 2)

                δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                A C

                Frsquo

                A C

                A B C

                24

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                Drsquo (A 0) (B 0) (C 0) (C 2)

                δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                A C

                Frsquo (A C (C 1)) (A B C (C 1))

                A C

                A B C

                25

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H

                Drsquo 3 5

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5)

                26

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5

                Drsquo 3 5

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5)

                G G

                35 35

                27

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5)

                G H

                35 3 4 5

                28

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5)

                H H

                3 4 5 3 4 5

                29

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                Drsquo 3 5 3 4 5 3 5 7

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5) (G 3 5 7)

                H G

                3 4 5 3 5 7

                30

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                Drsquo 3 5 3 4 5 3 5 7

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5) (G 3 5 7)

                H G

                3 4 5 3 5

                31

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                G G

                3 5 7 3 5 6

                32

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                G H

                3 5 7 3 4 5

                33

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                G G

                3 5 6 3 5

                34

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo

                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                G H

                3 5 6 3 4 5

                35

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                δlsquo G G G H H H H G H G

                ( ) (G 3 5)

                Frsquo (G 3 5 6)

                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                G H

                3 5 6 3 4 5

                36

                Build XFAs from Regex (Cont)

                bull Compile to XFAbull Finding efficient implementations

                37

                Experimental Results

                bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                38

                Experimental Results (Cont)

                39

                Experimental Results (Cont)

                40

                Conclusion

                bull DFAs for regular expressions often blow up when combined

                bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                bull Result compared to other feasible approaches reduce both time and space

                41

                • XFA Faster Signature Matching With Extended Automata
                • Outline
                • Introduction
                • Technical overview
                • Technical overview (Cont)
                • Technical overview (Cont) (2)
                • Technical overview (Cont) (3)
                • Technical overview (Cont) (4)
                • Build XFAs from Regex
                • Build XFAs from Regex (Cont)
                • Build XFAs from Regex (Cont) (2)
                • Build XFAs from Regex (Cont) (3)
                • Build XFAs from Regex (Cont) (4)
                • Build XFAs from Regex (Cont) (5)
                • Build XFAs from Regex (Cont) (6)
                • Build XFAs from Regex (Cont) (7)
                • Build XFAs from Regex (Cont) (8)
                • Build XFAs from Regex (Cont) (9)
                • Build XFAs from Regex (Cont) (10)
                • Build XFAs from Regex (Cont) (11)
                • Build XFAs from Regex (Cont) (12)
                • Build XFAs from Regex (Cont) (13)
                • Build XFAs from Regex (Cont) (14)
                • Build XFAs from Regex (Cont) (15)
                • Build XFAs from Regex (Cont) (16)
                • Build XFAs from Regex (Cont) (17)
                • Build XFAs from Regex (Cont) (18)
                • Build XFAs from Regex (Cont) (19)
                • Build XFAs from Regex (Cont) (20)
                • Build XFAs from Regex (Cont) (21)
                • Build XFAs from Regex (Cont) (22)
                • Build XFAs from Regex (Cont) (23)
                • Build XFAs from Regex (Cont) (24)
                • Build XFAs from Regex (Cont) (25)
                • Build XFAs from Regex (Cont) (26)
                • Build XFAs from Regex (Cont) (27)
                • Build XFAs from Regex (Cont) (28)
                • Experimental Results
                • Experimental Results (Cont)
                • Experimental Results (Cont) (2)
                • Conclusion

                  Build XFAs from Regex

                  bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

                  bull ε ndasheliminationbull determinizing transitionsbull data determinization

                  bull Finding efficient implementations

                  9

                  Build XFAs from Regex (Cont)

                  bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

                  bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

                  bull ex abcd =gt abcd

                  10

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Definitions

                  bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

                  data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

                  an initial data value d0bull F QtimesD is the set of accepting configurationssube

                  11

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Definitions

                  bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

                  transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

                  defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

                  12

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull From parse trees to NXFAs

                  13

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull From parse trees to NXFAs

                  bull Ex ab[^a]1 =gt ab[^a]1

                  14

                  ａ

                  ｂ

                  sum

                  1

                  [ a]

                  1

                  2

                  0

                  3

                  4

                  bit = 0cnt = 0 sum

                  a

                  bsum

                  [^a]

                  ε

                  ε

                  ε

                  cnt++

                  if (bit == 1 ampamp cnt = 1) accept()

                  bit = 1

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull From parse trees to NXFAs

                  bull Ex ab[^a]1 =gt ab[^a]1

                  15

                  1

                  2

                  0

                  3

                  4

                  bit = 0cnt = 0 sum

                  a

                  bsum

                  [^a]

                  ε

                  ε

                  ε

                  cnt++

                  if (bit == 1 ampamp cnt = 1) accept()

                  bit = 1

                  120576

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                  16

                  120576

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                  17

                  120576

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                  18

                  120576

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                  Qrsquo A C

                  Drsquo (A 0) (B 1) (C 0) (C 2)

                  δlsquo

                  A C

                  Frsquo

                  19

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                  Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                  Drsquo (A 0) (B 1) (C 0) (C 2)

                  δlsquo A C A C

                  A C

                  Frsquo

                  A C

                  A C

                  20

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                  δlsquo A C A C A C A B C

                  A C

                  Frsquo

                  A B C

                  A C

                  21

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                  δlsquo A C A C A C A B C A B C A B C

                  A C

                  Frsquo

                  A B C

                  A B C

                  22

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                  Drsquo (A 0) (B 0) (C 0) (C 2)

                  δlsquo A C A C A C A B C A B C A B C A B C A C

                  A C

                  Frsquo

                  A C

                  A B C

                  23

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                  Drsquo (A 0) (B 0) (C 0) (C 2)

                  δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                  A C

                  Frsquo

                  A C

                  A B C

                  24

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                  Drsquo (A 0) (B 0) (C 0) (C 2)

                  δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                  A C

                  Frsquo (A C (C 1)) (A B C (C 1))

                  A C

                  A B C

                  25

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H

                  Drsquo 3 5

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5)

                  26

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5

                  Drsquo 3 5

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5)

                  G G

                  35 35

                  27

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5)

                  G H

                  35 3 4 5

                  28

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5)

                  H H

                  3 4 5 3 4 5

                  29

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                  Drsquo 3 5 3 4 5 3 5 7

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5) (G 3 5 7)

                  H G

                  3 4 5 3 5 7

                  30

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                  Drsquo 3 5 3 4 5 3 5 7

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5) (G 3 5 7)

                  H G

                  3 4 5 3 5

                  31

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                  G G

                  3 5 7 3 5 6

                  32

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                  G H

                  3 5 7 3 4 5

                  33

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                  G G

                  3 5 6 3 5

                  34

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo

                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                  G H

                  3 5 6 3 4 5

                  35

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                  δlsquo G G G H H H H G H G

                  ( ) (G 3 5)

                  Frsquo (G 3 5 6)

                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                  G H

                  3 5 6 3 4 5

                  36

                  Build XFAs from Regex (Cont)

                  bull Compile to XFAbull Finding efficient implementations

                  37

                  Experimental Results

                  bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                  38

                  Experimental Results (Cont)

                  39

                  Experimental Results (Cont)

                  40

                  Conclusion

                  bull DFAs for regular expressions often blow up when combined

                  bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                  bull Result compared to other feasible approaches reduce both time and space

                  41

                  • XFA Faster Signature Matching With Extended Automata
                  • Outline
                  • Introduction
                  • Technical overview
                  • Technical overview (Cont)
                  • Technical overview (Cont) (2)
                  • Technical overview (Cont) (3)
                  • Technical overview (Cont) (4)
                  • Build XFAs from Regex
                  • Build XFAs from Regex (Cont)
                  • Build XFAs from Regex (Cont) (2)
                  • Build XFAs from Regex (Cont) (3)
                  • Build XFAs from Regex (Cont) (4)
                  • Build XFAs from Regex (Cont) (5)
                  • Build XFAs from Regex (Cont) (6)
                  • Build XFAs from Regex (Cont) (7)
                  • Build XFAs from Regex (Cont) (8)
                  • Build XFAs from Regex (Cont) (9)
                  • Build XFAs from Regex (Cont) (10)
                  • Build XFAs from Regex (Cont) (11)
                  • Build XFAs from Regex (Cont) (12)
                  • Build XFAs from Regex (Cont) (13)
                  • Build XFAs from Regex (Cont) (14)
                  • Build XFAs from Regex (Cont) (15)
                  • Build XFAs from Regex (Cont) (16)
                  • Build XFAs from Regex (Cont) (17)
                  • Build XFAs from Regex (Cont) (18)
                  • Build XFAs from Regex (Cont) (19)
                  • Build XFAs from Regex (Cont) (20)
                  • Build XFAs from Regex (Cont) (21)
                  • Build XFAs from Regex (Cont) (22)
                  • Build XFAs from Regex (Cont) (23)
                  • Build XFAs from Regex (Cont) (24)
                  • Build XFAs from Regex (Cont) (25)
                  • Build XFAs from Regex (Cont) (26)
                  • Build XFAs from Regex (Cont) (27)
                  • Build XFAs from Regex (Cont) (28)
                  • Experimental Results
                  • Experimental Results (Cont)
                  • Experimental Results (Cont) (2)
                  • Conclusion

                    Build XFAs from Regex (Cont)

                    bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

                    bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

                    bull ex abcd =gt abcd

                    10

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Definitions

                    bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

                    data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

                    an initial data value d0bull F QtimesD is the set of accepting configurationssube

                    11

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Definitions

                    bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

                    transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

                    defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

                    12

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull From parse trees to NXFAs

                    13

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull From parse trees to NXFAs

                    bull Ex ab[^a]1 =gt ab[^a]1

                    14

                    ａ

                    ｂ

                    sum

                    1

                    [ a]

                    1

                    2

                    0

                    3

                    4

                    bit = 0cnt = 0 sum

                    a

                    bsum

                    [^a]

                    ε

                    ε

                    ε

                    cnt++

                    if (bit == 1 ampamp cnt = 1) accept()

                    bit = 1

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull From parse trees to NXFAs

                    bull Ex ab[^a]1 =gt ab[^a]1

                    15

                    1

                    2

                    0

                    3

                    4

                    bit = 0cnt = 0 sum

                    a

                    bsum

                    [^a]

                    ε

                    ε

                    ε

                    cnt++

                    if (bit == 1 ampamp cnt = 1) accept()

                    bit = 1

                    120576

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                    16

                    120576

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                    17

                    120576

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                    18

                    120576

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                    Qrsquo A C

                    Drsquo (A 0) (B 1) (C 0) (C 2)

                    δlsquo

                    A C

                    Frsquo

                    19

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                    Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                    Drsquo (A 0) (B 1) (C 0) (C 2)

                    δlsquo A C A C

                    A C

                    Frsquo

                    A C

                    A C

                    20

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                    δlsquo A C A C A C A B C

                    A C

                    Frsquo

                    A B C

                    A C

                    21

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                    δlsquo A C A C A C A B C A B C A B C

                    A C

                    Frsquo

                    A B C

                    A B C

                    22

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                    Drsquo (A 0) (B 0) (C 0) (C 2)

                    δlsquo A C A C A C A B C A B C A B C A B C A C

                    A C

                    Frsquo

                    A C

                    A B C

                    23

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                    Drsquo (A 0) (B 0) (C 0) (C 2)

                    δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                    A C

                    Frsquo

                    A C

                    A B C

                    24

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                    Drsquo (A 0) (B 0) (C 0) (C 2)

                    δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                    A C

                    Frsquo (A C (C 1)) (A B C (C 1))

                    A C

                    A B C

                    25

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H

                    Drsquo 3 5

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5)

                    26

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5

                    Drsquo 3 5

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5)

                    G G

                    35 35

                    27

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5)

                    G H

                    35 3 4 5

                    28

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5)

                    H H

                    3 4 5 3 4 5

                    29

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                    Drsquo 3 5 3 4 5 3 5 7

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5) (G 3 5 7)

                    H G

                    3 4 5 3 5 7

                    30

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                    Drsquo 3 5 3 4 5 3 5 7

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5) (G 3 5 7)

                    H G

                    3 4 5 3 5

                    31

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                    G G

                    3 5 7 3 5 6

                    32

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                    G H

                    3 5 7 3 4 5

                    33

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                    G G

                    3 5 6 3 5

                    34

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo

                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                    G H

                    3 5 6 3 4 5

                    35

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                    δlsquo G G G H H H H G H G

                    ( ) (G 3 5)

                    Frsquo (G 3 5 6)

                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                    G H

                    3 5 6 3 4 5

                    36

                    Build XFAs from Regex (Cont)

                    bull Compile to XFAbull Finding efficient implementations

                    37

                    Experimental Results

                    bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                    38

                    Experimental Results (Cont)

                    39

                    Experimental Results (Cont)

                    40

                    Conclusion

                    bull DFAs for regular expressions often blow up when combined

                    bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                    bull Result compared to other feasible approaches reduce both time and space

                    41

                    • XFA Faster Signature Matching With Extended Automata
                    • Outline
                    • Introduction
                    • Technical overview
                    • Technical overview (Cont)
                    • Technical overview (Cont) (2)
                    • Technical overview (Cont) (3)
                    • Technical overview (Cont) (4)
                    • Build XFAs from Regex
                    • Build XFAs from Regex (Cont)
                    • Build XFAs from Regex (Cont) (2)
                    • Build XFAs from Regex (Cont) (3)
                    • Build XFAs from Regex (Cont) (4)
                    • Build XFAs from Regex (Cont) (5)
                    • Build XFAs from Regex (Cont) (6)
                    • Build XFAs from Regex (Cont) (7)
                    • Build XFAs from Regex (Cont) (8)
                    • Build XFAs from Regex (Cont) (9)
                    • Build XFAs from Regex (Cont) (10)
                    • Build XFAs from Regex (Cont) (11)
                    • Build XFAs from Regex (Cont) (12)
                    • Build XFAs from Regex (Cont) (13)
                    • Build XFAs from Regex (Cont) (14)
                    • Build XFAs from Regex (Cont) (15)
                    • Build XFAs from Regex (Cont) (16)
                    • Build XFAs from Regex (Cont) (17)
                    • Build XFAs from Regex (Cont) (18)
                    • Build XFAs from Regex (Cont) (19)
                    • Build XFAs from Regex (Cont) (20)
                    • Build XFAs from Regex (Cont) (21)
                    • Build XFAs from Regex (Cont) (22)
                    • Build XFAs from Regex (Cont) (23)
                    • Build XFAs from Regex (Cont) (24)
                    • Build XFAs from Regex (Cont) (25)
                    • Build XFAs from Regex (Cont) (26)
                    • Build XFAs from Regex (Cont) (27)
                    • Build XFAs from Regex (Cont) (28)
                    • Experimental Results
                    • Experimental Results (Cont)
                    • Experimental Results (Cont) (2)
                    • Conclusion

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Definitions

                      bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

                      data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

                      an initial data value d0bull F QtimesD is the set of accepting configurationssube

                      11

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Definitions

                      bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

                      transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

                      defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

                      12

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull From parse trees to NXFAs

                      13

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull From parse trees to NXFAs

                      bull Ex ab[^a]1 =gt ab[^a]1

                      14

                      ａ

                      ｂ

                      sum

                      1

                      [ a]

                      1

                      2

                      0

                      3

                      4

                      bit = 0cnt = 0 sum

                      a

                      bsum

                      [^a]

                      ε

                      ε

                      ε

                      cnt++

                      if (bit == 1 ampamp cnt = 1) accept()

                      bit = 1

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull From parse trees to NXFAs

                      bull Ex ab[^a]1 =gt ab[^a]1

                      15

                      1

                      2

                      0

                      3

                      4

                      bit = 0cnt = 0 sum

                      a

                      bsum

                      [^a]

                      ε

                      ε

                      ε

                      cnt++

                      if (bit == 1 ampamp cnt = 1) accept()

                      bit = 1

                      120576

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                      16

                      120576

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                      17

                      120576

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                      18

                      120576

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                      Qrsquo A C

                      Drsquo (A 0) (B 1) (C 0) (C 2)

                      δlsquo

                      A C

                      Frsquo

                      19

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                      Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                      Drsquo (A 0) (B 1) (C 0) (C 2)

                      δlsquo A C A C

                      A C

                      Frsquo

                      A C

                      A C

                      20

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                      δlsquo A C A C A C A B C

                      A C

                      Frsquo

                      A B C

                      A C

                      21

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                      δlsquo A C A C A C A B C A B C A B C

                      A C

                      Frsquo

                      A B C

                      A B C

                      22

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                      Drsquo (A 0) (B 0) (C 0) (C 2)

                      δlsquo A C A C A C A B C A B C A B C A B C A C

                      A C

                      Frsquo

                      A C

                      A B C

                      23

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                      Drsquo (A 0) (B 0) (C 0) (C 2)

                      δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                      A C

                      Frsquo

                      A C

                      A B C

                      24

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                      Drsquo (A 0) (B 0) (C 0) (C 2)

                      δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                      A C

                      Frsquo (A C (C 1)) (A B C (C 1))

                      A C

                      A B C

                      25

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H

                      Drsquo 3 5

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5)

                      26

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5

                      Drsquo 3 5

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5)

                      G G

                      35 35

                      27

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5)

                      G H

                      35 3 4 5

                      28

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5)

                      H H

                      3 4 5 3 4 5

                      29

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                      Drsquo 3 5 3 4 5 3 5 7

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5) (G 3 5 7)

                      H G

                      3 4 5 3 5 7

                      30

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                      Drsquo 3 5 3 4 5 3 5 7

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5) (G 3 5 7)

                      H G

                      3 4 5 3 5

                      31

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                      G G

                      3 5 7 3 5 6

                      32

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                      G H

                      3 5 7 3 4 5

                      33

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                      G G

                      3 5 6 3 5

                      34

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo

                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                      G H

                      3 5 6 3 4 5

                      35

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                      δlsquo G G G H H H H G H G

                      ( ) (G 3 5)

                      Frsquo (G 3 5 6)

                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                      G H

                      3 5 6 3 4 5

                      36

                      Build XFAs from Regex (Cont)

                      bull Compile to XFAbull Finding efficient implementations

                      37

                      Experimental Results

                      bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                      38

                      Experimental Results (Cont)

                      39

                      Experimental Results (Cont)

                      40

                      Conclusion

                      bull DFAs for regular expressions often blow up when combined

                      bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                      bull Result compared to other feasible approaches reduce both time and space

                      41

                      • XFA Faster Signature Matching With Extended Automata
                      • Outline
                      • Introduction
                      • Technical overview
                      • Technical overview (Cont)
                      • Technical overview (Cont) (2)
                      • Technical overview (Cont) (3)
                      • Technical overview (Cont) (4)
                      • Build XFAs from Regex
                      • Build XFAs from Regex (Cont)
                      • Build XFAs from Regex (Cont) (2)
                      • Build XFAs from Regex (Cont) (3)
                      • Build XFAs from Regex (Cont) (4)
                      • Build XFAs from Regex (Cont) (5)
                      • Build XFAs from Regex (Cont) (6)
                      • Build XFAs from Regex (Cont) (7)
                      • Build XFAs from Regex (Cont) (8)
                      • Build XFAs from Regex (Cont) (9)
                      • Build XFAs from Regex (Cont) (10)
                      • Build XFAs from Regex (Cont) (11)
                      • Build XFAs from Regex (Cont) (12)
                      • Build XFAs from Regex (Cont) (13)
                      • Build XFAs from Regex (Cont) (14)
                      • Build XFAs from Regex (Cont) (15)
                      • Build XFAs from Regex (Cont) (16)
                      • Build XFAs from Regex (Cont) (17)
                      • Build XFAs from Regex (Cont) (18)
                      • Build XFAs from Regex (Cont) (19)
                      • Build XFAs from Regex (Cont) (20)
                      • Build XFAs from Regex (Cont) (21)
                      • Build XFAs from Regex (Cont) (22)
                      • Build XFAs from Regex (Cont) (23)
                      • Build XFAs from Regex (Cont) (24)
                      • Build XFAs from Regex (Cont) (25)
                      • Build XFAs from Regex (Cont) (26)
                      • Build XFAs from Regex (Cont) (27)
                      • Build XFAs from Regex (Cont) (28)
                      • Experimental Results
                      • Experimental Results (Cont)
                      • Experimental Results (Cont) (2)
                      • Conclusion

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Definitions

                        bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

                        transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

                        defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

                        12

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull From parse trees to NXFAs

                        13

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull From parse trees to NXFAs

                        bull Ex ab[^a]1 =gt ab[^a]1

                        14

                        ａ

                        ｂ

                        sum

                        1

                        [ a]

                        1

                        2

                        0

                        3

                        4

                        bit = 0cnt = 0 sum

                        a

                        bsum

                        [^a]

                        ε

                        ε

                        ε

                        cnt++

                        if (bit == 1 ampamp cnt = 1) accept()

                        bit = 1

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull From parse trees to NXFAs

                        bull Ex ab[^a]1 =gt ab[^a]1

                        15

                        1

                        2

                        0

                        3

                        4

                        bit = 0cnt = 0 sum

                        a

                        bsum

                        [^a]

                        ε

                        ε

                        ε

                        cnt++

                        if (bit == 1 ampamp cnt = 1) accept()

                        bit = 1

                        120576

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                        16

                        120576

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                        17

                        120576

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                        18

                        120576

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                        Qrsquo A C

                        Drsquo (A 0) (B 1) (C 0) (C 2)

                        δlsquo

                        A C

                        Frsquo

                        19

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                        Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                        Drsquo (A 0) (B 1) (C 0) (C 2)

                        δlsquo A C A C

                        A C

                        Frsquo

                        A C

                        A C

                        20

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                        δlsquo A C A C A C A B C

                        A C

                        Frsquo

                        A B C

                        A C

                        21

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                        δlsquo A C A C A C A B C A B C A B C

                        A C

                        Frsquo

                        A B C

                        A B C

                        22

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                        Drsquo (A 0) (B 0) (C 0) (C 2)

                        δlsquo A C A C A C A B C A B C A B C A B C A C

                        A C

                        Frsquo

                        A C

                        A B C

                        23

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                        Drsquo (A 0) (B 0) (C 0) (C 2)

                        δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                        A C

                        Frsquo

                        A C

                        A B C

                        24

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                        Drsquo (A 0) (B 0) (C 0) (C 2)

                        δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                        A C

                        Frsquo (A C (C 1)) (A B C (C 1))

                        A C

                        A B C

                        25

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H

                        Drsquo 3 5

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5)

                        26

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5

                        Drsquo 3 5

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5)

                        G G

                        35 35

                        27

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5)

                        G H

                        35 3 4 5

                        28

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5)

                        H H

                        3 4 5 3 4 5

                        29

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                        Drsquo 3 5 3 4 5 3 5 7

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5) (G 3 5 7)

                        H G

                        3 4 5 3 5 7

                        30

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                        Drsquo 3 5 3 4 5 3 5 7

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5) (G 3 5 7)

                        H G

                        3 4 5 3 5

                        31

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                        G G

                        3 5 7 3 5 6

                        32

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                        G H

                        3 5 7 3 4 5

                        33

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                        G G

                        3 5 6 3 5

                        34

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo

                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                        G H

                        3 5 6 3 4 5

                        35

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                        δlsquo G G G H H H H G H G

                        ( ) (G 3 5)

                        Frsquo (G 3 5 6)

                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                        G H

                        3 5 6 3 4 5

                        36

                        Build XFAs from Regex (Cont)

                        bull Compile to XFAbull Finding efficient implementations

                        37

                        Experimental Results

                        bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                        38

                        Experimental Results (Cont)

                        39

                        Experimental Results (Cont)

                        40

                        Conclusion

                        bull DFAs for regular expressions often blow up when combined

                        bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                        bull Result compared to other feasible approaches reduce both time and space

                        41

                        • XFA Faster Signature Matching With Extended Automata
                        • Outline
                        • Introduction
                        • Technical overview
                        • Technical overview (Cont)
                        • Technical overview (Cont) (2)
                        • Technical overview (Cont) (3)
                        • Technical overview (Cont) (4)
                        • Build XFAs from Regex
                        • Build XFAs from Regex (Cont)
                        • Build XFAs from Regex (Cont) (2)
                        • Build XFAs from Regex (Cont) (3)
                        • Build XFAs from Regex (Cont) (4)
                        • Build XFAs from Regex (Cont) (5)
                        • Build XFAs from Regex (Cont) (6)
                        • Build XFAs from Regex (Cont) (7)
                        • Build XFAs from Regex (Cont) (8)
                        • Build XFAs from Regex (Cont) (9)
                        • Build XFAs from Regex (Cont) (10)
                        • Build XFAs from Regex (Cont) (11)
                        • Build XFAs from Regex (Cont) (12)
                        • Build XFAs from Regex (Cont) (13)
                        • Build XFAs from Regex (Cont) (14)
                        • Build XFAs from Regex (Cont) (15)
                        • Build XFAs from Regex (Cont) (16)
                        • Build XFAs from Regex (Cont) (17)
                        • Build XFAs from Regex (Cont) (18)
                        • Build XFAs from Regex (Cont) (19)
                        • Build XFAs from Regex (Cont) (20)
                        • Build XFAs from Regex (Cont) (21)
                        • Build XFAs from Regex (Cont) (22)
                        • Build XFAs from Regex (Cont) (23)
                        • Build XFAs from Regex (Cont) (24)
                        • Build XFAs from Regex (Cont) (25)
                        • Build XFAs from Regex (Cont) (26)
                        • Build XFAs from Regex (Cont) (27)
                        • Build XFAs from Regex (Cont) (28)
                        • Experimental Results
                        • Experimental Results (Cont)
                        • Experimental Results (Cont) (2)
                        • Conclusion

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull From parse trees to NXFAs

                          13

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull From parse trees to NXFAs

                          bull Ex ab[^a]1 =gt ab[^a]1

                          14

                          ａ

                          ｂ

                          sum

                          1

                          [ a]

                          1

                          2

                          0

                          3

                          4

                          bit = 0cnt = 0 sum

                          a

                          bsum

                          [^a]

                          ε

                          ε

                          ε

                          cnt++

                          if (bit == 1 ampamp cnt = 1) accept()

                          bit = 1

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull From parse trees to NXFAs

                          bull Ex ab[^a]1 =gt ab[^a]1

                          15

                          1

                          2

                          0

                          3

                          4

                          bit = 0cnt = 0 sum

                          a

                          bsum

                          [^a]

                          ε

                          ε

                          ε

                          cnt++

                          if (bit == 1 ampamp cnt = 1) accept()

                          bit = 1

                          120576

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                          16

                          120576

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                          17

                          120576

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                          18

                          120576

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                          Qrsquo A C

                          Drsquo (A 0) (B 1) (C 0) (C 2)

                          δlsquo

                          A C

                          Frsquo

                          19

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                          Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                          Drsquo (A 0) (B 1) (C 0) (C 2)

                          δlsquo A C A C

                          A C

                          Frsquo

                          A C

                          A C

                          20

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                          δlsquo A C A C A C A B C

                          A C

                          Frsquo

                          A B C

                          A C

                          21

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                          δlsquo A C A C A C A B C A B C A B C

                          A C

                          Frsquo

                          A B C

                          A B C

                          22

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                          Drsquo (A 0) (B 0) (C 0) (C 2)

                          δlsquo A C A C A C A B C A B C A B C A B C A C

                          A C

                          Frsquo

                          A C

                          A B C

                          23

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                          Drsquo (A 0) (B 0) (C 0) (C 2)

                          δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                          A C

                          Frsquo

                          A C

                          A B C

                          24

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                          Drsquo (A 0) (B 0) (C 0) (C 2)

                          δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                          A C

                          Frsquo (A C (C 1)) (A B C (C 1))

                          A C

                          A B C

                          25

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H

                          Drsquo 3 5

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5)

                          26

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5

                          Drsquo 3 5

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5)

                          G G

                          35 35

                          27

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5)

                          G H

                          35 3 4 5

                          28

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5)

                          H H

                          3 4 5 3 4 5

                          29

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                          Drsquo 3 5 3 4 5 3 5 7

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5) (G 3 5 7)

                          H G

                          3 4 5 3 5 7

                          30

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                          Drsquo 3 5 3 4 5 3 5 7

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5) (G 3 5 7)

                          H G

                          3 4 5 3 5

                          31

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                          G G

                          3 5 7 3 5 6

                          32

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                          G H

                          3 5 7 3 4 5

                          33

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                          G G

                          3 5 6 3 5

                          34

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo

                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                          G H

                          3 5 6 3 4 5

                          35

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                          δlsquo G G G H H H H G H G

                          ( ) (G 3 5)

                          Frsquo (G 3 5 6)

                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                          G H

                          3 5 6 3 4 5

                          36

                          Build XFAs from Regex (Cont)

                          bull Compile to XFAbull Finding efficient implementations

                          37

                          Experimental Results

                          bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                          38

                          Experimental Results (Cont)

                          39

                          Experimental Results (Cont)

                          40

                          Conclusion

                          bull DFAs for regular expressions often blow up when combined

                          bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                          bull Result compared to other feasible approaches reduce both time and space

                          41

                          • XFA Faster Signature Matching With Extended Automata
                          • Outline
                          • Introduction
                          • Technical overview
                          • Technical overview (Cont)
                          • Technical overview (Cont) (2)
                          • Technical overview (Cont) (3)
                          • Technical overview (Cont) (4)
                          • Build XFAs from Regex
                          • Build XFAs from Regex (Cont)
                          • Build XFAs from Regex (Cont) (2)
                          • Build XFAs from Regex (Cont) (3)
                          • Build XFAs from Regex (Cont) (4)
                          • Build XFAs from Regex (Cont) (5)
                          • Build XFAs from Regex (Cont) (6)
                          • Build XFAs from Regex (Cont) (7)
                          • Build XFAs from Regex (Cont) (8)
                          • Build XFAs from Regex (Cont) (9)
                          • Build XFAs from Regex (Cont) (10)
                          • Build XFAs from Regex (Cont) (11)
                          • Build XFAs from Regex (Cont) (12)
                          • Build XFAs from Regex (Cont) (13)
                          • Build XFAs from Regex (Cont) (14)
                          • Build XFAs from Regex (Cont) (15)
                          • Build XFAs from Regex (Cont) (16)
                          • Build XFAs from Regex (Cont) (17)
                          • Build XFAs from Regex (Cont) (18)
                          • Build XFAs from Regex (Cont) (19)
                          • Build XFAs from Regex (Cont) (20)
                          • Build XFAs from Regex (Cont) (21)
                          • Build XFAs from Regex (Cont) (22)
                          • Build XFAs from Regex (Cont) (23)
                          • Build XFAs from Regex (Cont) (24)
                          • Build XFAs from Regex (Cont) (25)
                          • Build XFAs from Regex (Cont) (26)
                          • Build XFAs from Regex (Cont) (27)
                          • Build XFAs from Regex (Cont) (28)
                          • Experimental Results
                          • Experimental Results (Cont)
                          • Experimental Results (Cont) (2)
                          • Conclusion

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull From parse trees to NXFAs

                            bull Ex ab[^a]1 =gt ab[^a]1

                            14

                            ａ

                            ｂ

                            sum

                            1

                            [ a]

                            1

                            2

                            0

                            3

                            4

                            bit = 0cnt = 0 sum

                            a

                            bsum

                            [^a]

                            ε

                            ε

                            ε

                            cnt++

                            if (bit == 1 ampamp cnt = 1) accept()

                            bit = 1

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull From parse trees to NXFAs

                            bull Ex ab[^a]1 =gt ab[^a]1

                            15

                            1

                            2

                            0

                            3

                            4

                            bit = 0cnt = 0 sum

                            a

                            bsum

                            [^a]

                            ε

                            ε

                            ε

                            cnt++

                            if (bit == 1 ampamp cnt = 1) accept()

                            bit = 1

                            120576

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                            16

                            120576

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                            17

                            120576

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                            18

                            120576

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                            Qrsquo A C

                            Drsquo (A 0) (B 1) (C 0) (C 2)

                            δlsquo

                            A C

                            Frsquo

                            19

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                            Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                            Drsquo (A 0) (B 1) (C 0) (C 2)

                            δlsquo A C A C

                            A C

                            Frsquo

                            A C

                            A C

                            20

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                            δlsquo A C A C A C A B C

                            A C

                            Frsquo

                            A B C

                            A C

                            21

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                            δlsquo A C A C A C A B C A B C A B C

                            A C

                            Frsquo

                            A B C

                            A B C

                            22

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                            Drsquo (A 0) (B 0) (C 0) (C 2)

                            δlsquo A C A C A C A B C A B C A B C A B C A C

                            A C

                            Frsquo

                            A C

                            A B C

                            23

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                            Drsquo (A 0) (B 0) (C 0) (C 2)

                            δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                            A C

                            Frsquo

                            A C

                            A B C

                            24

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                            Drsquo (A 0) (B 0) (C 0) (C 2)

                            δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                            A C

                            Frsquo (A C (C 1)) (A B C (C 1))

                            A C

                            A B C

                            25

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H

                            Drsquo 3 5

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5)

                            26

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5

                            Drsquo 3 5

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5)

                            G G

                            35 35

                            27

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5)

                            G H

                            35 3 4 5

                            28

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5)

                            H H

                            3 4 5 3 4 5

                            29

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                            Drsquo 3 5 3 4 5 3 5 7

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5) (G 3 5 7)

                            H G

                            3 4 5 3 5 7

                            30

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                            Drsquo 3 5 3 4 5 3 5 7

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5) (G 3 5 7)

                            H G

                            3 4 5 3 5

                            31

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                            G G

                            3 5 7 3 5 6

                            32

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                            G H

                            3 5 7 3 4 5

                            33

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                            G G

                            3 5 6 3 5

                            34

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo

                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                            G H

                            3 5 6 3 4 5

                            35

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                            δlsquo G G G H H H H G H G

                            ( ) (G 3 5)

                            Frsquo (G 3 5 6)

                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                            G H

                            3 5 6 3 4 5

                            36

                            Build XFAs from Regex (Cont)

                            bull Compile to XFAbull Finding efficient implementations

                            37

                            Experimental Results

                            bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                            38

                            Experimental Results (Cont)

                            39

                            Experimental Results (Cont)

                            40

                            Conclusion

                            bull DFAs for regular expressions often blow up when combined

                            bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                            bull Result compared to other feasible approaches reduce both time and space

                            41

                            • XFA Faster Signature Matching With Extended Automata
                            • Outline
                            • Introduction
                            • Technical overview
                            • Technical overview (Cont)
                            • Technical overview (Cont) (2)
                            • Technical overview (Cont) (3)
                            • Technical overview (Cont) (4)
                            • Build XFAs from Regex
                            • Build XFAs from Regex (Cont)
                            • Build XFAs from Regex (Cont) (2)
                            • Build XFAs from Regex (Cont) (3)
                            • Build XFAs from Regex (Cont) (4)
                            • Build XFAs from Regex (Cont) (5)
                            • Build XFAs from Regex (Cont) (6)
                            • Build XFAs from Regex (Cont) (7)
                            • Build XFAs from Regex (Cont) (8)
                            • Build XFAs from Regex (Cont) (9)
                            • Build XFAs from Regex (Cont) (10)
                            • Build XFAs from Regex (Cont) (11)
                            • Build XFAs from Regex (Cont) (12)
                            • Build XFAs from Regex (Cont) (13)
                            • Build XFAs from Regex (Cont) (14)
                            • Build XFAs from Regex (Cont) (15)
                            • Build XFAs from Regex (Cont) (16)
                            • Build XFAs from Regex (Cont) (17)
                            • Build XFAs from Regex (Cont) (18)
                            • Build XFAs from Regex (Cont) (19)
                            • Build XFAs from Regex (Cont) (20)
                            • Build XFAs from Regex (Cont) (21)
                            • Build XFAs from Regex (Cont) (22)
                            • Build XFAs from Regex (Cont) (23)
                            • Build XFAs from Regex (Cont) (24)
                            • Build XFAs from Regex (Cont) (25)
                            • Build XFAs from Regex (Cont) (26)
                            • Build XFAs from Regex (Cont) (27)
                            • Build XFAs from Regex (Cont) (28)
                            • Experimental Results
                            • Experimental Results (Cont)
                            • Experimental Results (Cont) (2)
                            • Conclusion

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull From parse trees to NXFAs

                              bull Ex ab[^a]1 =gt ab[^a]1

                              15

                              1

                              2

                              0

                              3

                              4

                              bit = 0cnt = 0 sum

                              a

                              bsum

                              [^a]

                              ε

                              ε

                              ε

                              cnt++

                              if (bit == 1 ampamp cnt = 1) accept()

                              bit = 1

                              120576

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                              16

                              120576

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                              17

                              120576

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                              18

                              120576

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                              Qrsquo A C

                              Drsquo (A 0) (B 1) (C 0) (C 2)

                              δlsquo

                              A C

                              Frsquo

                              19

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                              Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                              Drsquo (A 0) (B 1) (C 0) (C 2)

                              δlsquo A C A C

                              A C

                              Frsquo

                              A C

                              A C

                              20

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                              δlsquo A C A C A C A B C

                              A C

                              Frsquo

                              A B C

                              A C

                              21

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                              δlsquo A C A C A C A B C A B C A B C

                              A C

                              Frsquo

                              A B C

                              A B C

                              22

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                              Drsquo (A 0) (B 0) (C 0) (C 2)

                              δlsquo A C A C A C A B C A B C A B C A B C A C

                              A C

                              Frsquo

                              A C

                              A B C

                              23

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                              Drsquo (A 0) (B 0) (C 0) (C 2)

                              δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                              A C

                              Frsquo

                              A C

                              A B C

                              24

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                              Drsquo (A 0) (B 0) (C 0) (C 2)

                              δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                              A C

                              Frsquo (A C (C 1)) (A B C (C 1))

                              A C

                              A B C

                              25

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H

                              Drsquo 3 5

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5)

                              26

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5

                              Drsquo 3 5

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5)

                              G G

                              35 35

                              27

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5)

                              G H

                              35 3 4 5

                              28

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5)

                              H H

                              3 4 5 3 4 5

                              29

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                              Drsquo 3 5 3 4 5 3 5 7

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5) (G 3 5 7)

                              H G

                              3 4 5 3 5 7

                              30

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                              Drsquo 3 5 3 4 5 3 5 7

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5) (G 3 5 7)

                              H G

                              3 4 5 3 5

                              31

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                              G G

                              3 5 7 3 5 6

                              32

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                              G H

                              3 5 7 3 4 5

                              33

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                              G G

                              3 5 6 3 5

                              34

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo

                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                              G H

                              3 5 6 3 4 5

                              35

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                              δlsquo G G G H H H H G H G

                              ( ) (G 3 5)

                              Frsquo (G 3 5 6)

                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                              G H

                              3 5 6 3 4 5

                              36

                              Build XFAs from Regex (Cont)

                              bull Compile to XFAbull Finding efficient implementations

                              37

                              Experimental Results

                              bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                              38

                              Experimental Results (Cont)

                              39

                              Experimental Results (Cont)

                              40

                              Conclusion

                              bull DFAs for regular expressions often blow up when combined

                              bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                              bull Result compared to other feasible approaches reduce both time and space

                              41

                              • XFA Faster Signature Matching With Extended Automata
                              • Outline
                              • Introduction
                              • Technical overview
                              • Technical overview (Cont)
                              • Technical overview (Cont) (2)
                              • Technical overview (Cont) (3)
                              • Technical overview (Cont) (4)
                              • Build XFAs from Regex
                              • Build XFAs from Regex (Cont)
                              • Build XFAs from Regex (Cont) (2)
                              • Build XFAs from Regex (Cont) (3)
                              • Build XFAs from Regex (Cont) (4)
                              • Build XFAs from Regex (Cont) (5)
                              • Build XFAs from Regex (Cont) (6)
                              • Build XFAs from Regex (Cont) (7)
                              • Build XFAs from Regex (Cont) (8)
                              • Build XFAs from Regex (Cont) (9)
                              • Build XFAs from Regex (Cont) (10)
                              • Build XFAs from Regex (Cont) (11)
                              • Build XFAs from Regex (Cont) (12)
                              • Build XFAs from Regex (Cont) (13)
                              • Build XFAs from Regex (Cont) (14)
                              • Build XFAs from Regex (Cont) (15)
                              • Build XFAs from Regex (Cont) (16)
                              • Build XFAs from Regex (Cont) (17)
                              • Build XFAs from Regex (Cont) (18)
                              • Build XFAs from Regex (Cont) (19)
                              • Build XFAs from Regex (Cont) (20)
                              • Build XFAs from Regex (Cont) (21)
                              • Build XFAs from Regex (Cont) (22)
                              • Build XFAs from Regex (Cont) (23)
                              • Build XFAs from Regex (Cont) (24)
                              • Build XFAs from Regex (Cont) (25)
                              • Build XFAs from Regex (Cont) (26)
                              • Build XFAs from Regex (Cont) (27)
                              • Build XFAs from Regex (Cont) (28)
                              • Experimental Results
                              • Experimental Results (Cont)
                              • Experimental Results (Cont) (2)
                              • Conclusion

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                                16

                                120576

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                                17

                                120576

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                                18

                                120576

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                Qrsquo A C

                                Drsquo (A 0) (B 1) (C 0) (C 2)

                                δlsquo

                                A C

                                Frsquo

                                19

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                Drsquo (A 0) (B 1) (C 0) (C 2)

                                δlsquo A C A C

                                A C

                                Frsquo

                                A C

                                A C

                                20

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                                δlsquo A C A C A C A B C

                                A C

                                Frsquo

                                A B C

                                A C

                                21

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                                δlsquo A C A C A C A B C A B C A B C

                                A C

                                Frsquo

                                A B C

                                A B C

                                22

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                Drsquo (A 0) (B 0) (C 0) (C 2)

                                δlsquo A C A C A C A B C A B C A B C A B C A C

                                A C

                                Frsquo

                                A C

                                A B C

                                23

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                Drsquo (A 0) (B 0) (C 0) (C 2)

                                δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                A C

                                Frsquo

                                A C

                                A B C

                                24

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                Drsquo (A 0) (B 0) (C 0) (C 2)

                                δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                A C

                                Frsquo (A C (C 1)) (A B C (C 1))

                                A C

                                A B C

                                25

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H

                                Drsquo 3 5

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5)

                                26

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5

                                Drsquo 3 5

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5)

                                G G

                                35 35

                                27

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5)

                                G H

                                35 3 4 5

                                28

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5)

                                H H

                                3 4 5 3 4 5

                                29

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                Drsquo 3 5 3 4 5 3 5 7

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                H G

                                3 4 5 3 5 7

                                30

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                Drsquo 3 5 3 4 5 3 5 7

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                H G

                                3 4 5 3 5

                                31

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                G G

                                3 5 7 3 5 6

                                32

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                G H

                                3 5 7 3 4 5

                                33

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                G G

                                3 5 6 3 5

                                34

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo

                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                G H

                                3 5 6 3 4 5

                                35

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                δlsquo G G G H H H H G H G

                                ( ) (G 3 5)

                                Frsquo (G 3 5 6)

                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                G H

                                3 5 6 3 4 5

                                36

                                Build XFAs from Regex (Cont)

                                bull Compile to XFAbull Finding efficient implementations

                                37

                                Experimental Results

                                bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                38

                                Experimental Results (Cont)

                                39

                                Experimental Results (Cont)

                                40

                                Conclusion

                                bull DFAs for regular expressions often blow up when combined

                                bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                bull Result compared to other feasible approaches reduce both time and space

                                41

                                • XFA Faster Signature Matching With Extended Automata
                                • Outline
                                • Introduction
                                • Technical overview
                                • Technical overview (Cont)
                                • Technical overview (Cont) (2)
                                • Technical overview (Cont) (3)
                                • Technical overview (Cont) (4)
                                • Build XFAs from Regex
                                • Build XFAs from Regex (Cont)
                                • Build XFAs from Regex (Cont) (2)
                                • Build XFAs from Regex (Cont) (3)
                                • Build XFAs from Regex (Cont) (4)
                                • Build XFAs from Regex (Cont) (5)
                                • Build XFAs from Regex (Cont) (6)
                                • Build XFAs from Regex (Cont) (7)
                                • Build XFAs from Regex (Cont) (8)
                                • Build XFAs from Regex (Cont) (9)
                                • Build XFAs from Regex (Cont) (10)
                                • Build XFAs from Regex (Cont) (11)
                                • Build XFAs from Regex (Cont) (12)
                                • Build XFAs from Regex (Cont) (13)
                                • Build XFAs from Regex (Cont) (14)
                                • Build XFAs from Regex (Cont) (15)
                                • Build XFAs from Regex (Cont) (16)
                                • Build XFAs from Regex (Cont) (17)
                                • Build XFAs from Regex (Cont) (18)
                                • Build XFAs from Regex (Cont) (19)
                                • Build XFAs from Regex (Cont) (20)
                                • Build XFAs from Regex (Cont) (21)
                                • Build XFAs from Regex (Cont) (22)
                                • Build XFAs from Regex (Cont) (23)
                                • Build XFAs from Regex (Cont) (24)
                                • Build XFAs from Regex (Cont) (25)
                                • Build XFAs from Regex (Cont) (26)
                                • Build XFAs from Regex (Cont) (27)
                                • Build XFAs from Regex (Cont) (28)
                                • Experimental Results
                                • Experimental Results (Cont)
                                • Experimental Results (Cont) (2)
                                • Conclusion

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                                  17

                                  120576

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                                  18

                                  120576

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                  Qrsquo A C

                                  Drsquo (A 0) (B 1) (C 0) (C 2)

                                  δlsquo

                                  A C

                                  Frsquo

                                  19

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                  Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                  Drsquo (A 0) (B 1) (C 0) (C 2)

                                  δlsquo A C A C

                                  A C

                                  Frsquo

                                  A C

                                  A C

                                  20

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                                  δlsquo A C A C A C A B C

                                  A C

                                  Frsquo

                                  A B C

                                  A C

                                  21

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                                  δlsquo A C A C A C A B C A B C A B C

                                  A C

                                  Frsquo

                                  A B C

                                  A B C

                                  22

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                  Drsquo (A 0) (B 0) (C 0) (C 2)

                                  δlsquo A C A C A C A B C A B C A B C A B C A C

                                  A C

                                  Frsquo

                                  A C

                                  A B C

                                  23

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                  Drsquo (A 0) (B 0) (C 0) (C 2)

                                  δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                  A C

                                  Frsquo

                                  A C

                                  A B C

                                  24

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                  Drsquo (A 0) (B 0) (C 0) (C 2)

                                  δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                  A C

                                  Frsquo (A C (C 1)) (A B C (C 1))

                                  A C

                                  A B C

                                  25

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H

                                  Drsquo 3 5

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5)

                                  26

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5

                                  Drsquo 3 5

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5)

                                  G G

                                  35 35

                                  27

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5)

                                  G H

                                  35 3 4 5

                                  28

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5)

                                  H H

                                  3 4 5 3 4 5

                                  29

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                  Drsquo 3 5 3 4 5 3 5 7

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                  H G

                                  3 4 5 3 5 7

                                  30

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                  Drsquo 3 5 3 4 5 3 5 7

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                  H G

                                  3 4 5 3 5

                                  31

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                  G G

                                  3 5 7 3 5 6

                                  32

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                  G H

                                  3 5 7 3 4 5

                                  33

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                  G G

                                  3 5 6 3 5

                                  34

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo

                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                  G H

                                  3 5 6 3 4 5

                                  35

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                  δlsquo G G G H H H H G H G

                                  ( ) (G 3 5)

                                  Frsquo (G 3 5 6)

                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                  G H

                                  3 5 6 3 4 5

                                  36

                                  Build XFAs from Regex (Cont)

                                  bull Compile to XFAbull Finding efficient implementations

                                  37

                                  Experimental Results

                                  bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                  38

                                  Experimental Results (Cont)

                                  39

                                  Experimental Results (Cont)

                                  40

                                  Conclusion

                                  bull DFAs for regular expressions often blow up when combined

                                  bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                  bull Result compared to other feasible approaches reduce both time and space

                                  41

                                  • XFA Faster Signature Matching With Extended Automata
                                  • Outline
                                  • Introduction
                                  • Technical overview
                                  • Technical overview (Cont)
                                  • Technical overview (Cont) (2)
                                  • Technical overview (Cont) (3)
                                  • Technical overview (Cont) (4)
                                  • Build XFAs from Regex
                                  • Build XFAs from Regex (Cont)
                                  • Build XFAs from Regex (Cont) (2)
                                  • Build XFAs from Regex (Cont) (3)
                                  • Build XFAs from Regex (Cont) (4)
                                  • Build XFAs from Regex (Cont) (5)
                                  • Build XFAs from Regex (Cont) (6)
                                  • Build XFAs from Regex (Cont) (7)
                                  • Build XFAs from Regex (Cont) (8)
                                  • Build XFAs from Regex (Cont) (9)
                                  • Build XFAs from Regex (Cont) (10)
                                  • Build XFAs from Regex (Cont) (11)
                                  • Build XFAs from Regex (Cont) (12)
                                  • Build XFAs from Regex (Cont) (13)
                                  • Build XFAs from Regex (Cont) (14)
                                  • Build XFAs from Regex (Cont) (15)
                                  • Build XFAs from Regex (Cont) (16)
                                  • Build XFAs from Regex (Cont) (17)
                                  • Build XFAs from Regex (Cont) (18)
                                  • Build XFAs from Regex (Cont) (19)
                                  • Build XFAs from Regex (Cont) (20)
                                  • Build XFAs from Regex (Cont) (21)
                                  • Build XFAs from Regex (Cont) (22)
                                  • Build XFAs from Regex (Cont) (23)
                                  • Build XFAs from Regex (Cont) (24)
                                  • Build XFAs from Regex (Cont) (25)
                                  • Build XFAs from Regex (Cont) (26)
                                  • Build XFAs from Regex (Cont) (27)
                                  • Build XFAs from Regex (Cont) (28)
                                  • Experimental Results
                                  • Experimental Results (Cont)
                                  • Experimental Results (Cont) (2)
                                  • Conclusion

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

                                    18

                                    120576

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                    Qrsquo A C

                                    Drsquo (A 0) (B 1) (C 0) (C 2)

                                    δlsquo

                                    A C

                                    Frsquo

                                    19

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                    Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                    Drsquo (A 0) (B 1) (C 0) (C 2)

                                    δlsquo A C A C

                                    A C

                                    Frsquo

                                    A C

                                    A C

                                    20

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                                    δlsquo A C A C A C A B C

                                    A C

                                    Frsquo

                                    A B C

                                    A C

                                    21

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                                    δlsquo A C A C A C A B C A B C A B C

                                    A C

                                    Frsquo

                                    A B C

                                    A B C

                                    22

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                    Drsquo (A 0) (B 0) (C 0) (C 2)

                                    δlsquo A C A C A C A B C A B C A B C A B C A C

                                    A C

                                    Frsquo

                                    A C

                                    A B C

                                    23

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                    Drsquo (A 0) (B 0) (C 0) (C 2)

                                    δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                    A C

                                    Frsquo

                                    A C

                                    A B C

                                    24

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                    Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                    Drsquo (A 0) (B 0) (C 0) (C 2)

                                    δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                    A C

                                    Frsquo (A C (C 1)) (A B C (C 1))

                                    A C

                                    A B C

                                    25

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H

                                    Drsquo 3 5

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5)

                                    26

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5

                                    Drsquo 3 5

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5)

                                    G G

                                    35 35

                                    27

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5)

                                    G H

                                    35 3 4 5

                                    28

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5)

                                    H H

                                    3 4 5 3 4 5

                                    29

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                    Drsquo 3 5 3 4 5 3 5 7

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                    H G

                                    3 4 5 3 5 7

                                    30

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                    Drsquo 3 5 3 4 5 3 5 7

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                    H G

                                    3 4 5 3 5

                                    31

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                    G G

                                    3 5 7 3 5 6

                                    32

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                    G H

                                    3 5 7 3 4 5

                                    33

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                    G G

                                    3 5 6 3 5

                                    34

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo

                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                    G H

                                    3 5 6 3 4 5

                                    35

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                    δlsquo G G G H H H H G H G

                                    ( ) (G 3 5)

                                    Frsquo (G 3 5 6)

                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                    G H

                                    3 5 6 3 4 5

                                    36

                                    Build XFAs from Regex (Cont)

                                    bull Compile to XFAbull Finding efficient implementations

                                    37

                                    Experimental Results

                                    bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                    38

                                    Experimental Results (Cont)

                                    39

                                    Experimental Results (Cont)

                                    40

                                    Conclusion

                                    bull DFAs for regular expressions often blow up when combined

                                    bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                    bull Result compared to other feasible approaches reduce both time and space

                                    41

                                    • XFA Faster Signature Matching With Extended Automata
                                    • Outline
                                    • Introduction
                                    • Technical overview
                                    • Technical overview (Cont)
                                    • Technical overview (Cont) (2)
                                    • Technical overview (Cont) (3)
                                    • Technical overview (Cont) (4)
                                    • Build XFAs from Regex
                                    • Build XFAs from Regex (Cont)
                                    • Build XFAs from Regex (Cont) (2)
                                    • Build XFAs from Regex (Cont) (3)
                                    • Build XFAs from Regex (Cont) (4)
                                    • Build XFAs from Regex (Cont) (5)
                                    • Build XFAs from Regex (Cont) (6)
                                    • Build XFAs from Regex (Cont) (7)
                                    • Build XFAs from Regex (Cont) (8)
                                    • Build XFAs from Regex (Cont) (9)
                                    • Build XFAs from Regex (Cont) (10)
                                    • Build XFAs from Regex (Cont) (11)
                                    • Build XFAs from Regex (Cont) (12)
                                    • Build XFAs from Regex (Cont) (13)
                                    • Build XFAs from Regex (Cont) (14)
                                    • Build XFAs from Regex (Cont) (15)
                                    • Build XFAs from Regex (Cont) (16)
                                    • Build XFAs from Regex (Cont) (17)
                                    • Build XFAs from Regex (Cont) (18)
                                    • Build XFAs from Regex (Cont) (19)
                                    • Build XFAs from Regex (Cont) (20)
                                    • Build XFAs from Regex (Cont) (21)
                                    • Build XFAs from Regex (Cont) (22)
                                    • Build XFAs from Regex (Cont) (23)
                                    • Build XFAs from Regex (Cont) (24)
                                    • Build XFAs from Regex (Cont) (25)
                                    • Build XFAs from Regex (Cont) (26)
                                    • Build XFAs from Regex (Cont) (27)
                                    • Build XFAs from Regex (Cont) (28)
                                    • Experimental Results
                                    • Experimental Results (Cont)
                                    • Experimental Results (Cont) (2)
                                    • Conclusion

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                      Qrsquo A C

                                      Drsquo (A 0) (B 1) (C 0) (C 2)

                                      δlsquo

                                      A C

                                      Frsquo

                                      19

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                      Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                      Drsquo (A 0) (B 1) (C 0) (C 2)

                                      δlsquo A C A C

                                      A C

                                      Frsquo

                                      A C

                                      A C

                                      20

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                                      δlsquo A C A C A C A B C

                                      A C

                                      Frsquo

                                      A B C

                                      A C

                                      21

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                                      δlsquo A C A C A C A B C A B C A B C

                                      A C

                                      Frsquo

                                      A B C

                                      A B C

                                      22

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                      Drsquo (A 0) (B 0) (C 0) (C 2)

                                      δlsquo A C A C A C A B C A B C A B C A B C A C

                                      A C

                                      Frsquo

                                      A C

                                      A B C

                                      23

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                      Drsquo (A 0) (B 0) (C 0) (C 2)

                                      δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                      A C

                                      Frsquo

                                      A C

                                      A B C

                                      24

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                      Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                      Drsquo (A 0) (B 0) (C 0) (C 2)

                                      δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                      A C

                                      Frsquo (A C (C 1)) (A B C (C 1))

                                      A C

                                      A B C

                                      25

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H

                                      Drsquo 3 5

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5)

                                      26

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5

                                      Drsquo 3 5

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5)

                                      G G

                                      35 35

                                      27

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5)

                                      G H

                                      35 3 4 5

                                      28

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5)

                                      H H

                                      3 4 5 3 4 5

                                      29

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                      Drsquo 3 5 3 4 5 3 5 7

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                      H G

                                      3 4 5 3 5 7

                                      30

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                      Drsquo 3 5 3 4 5 3 5 7

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                      H G

                                      3 4 5 3 5

                                      31

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                      G G

                                      3 5 7 3 5 6

                                      32

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                      G H

                                      3 5 7 3 4 5

                                      33

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                      G G

                                      3 5 6 3 5

                                      34

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo

                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                      G H

                                      3 5 6 3 4 5

                                      35

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                      δlsquo G G G H H H H G H G

                                      ( ) (G 3 5)

                                      Frsquo (G 3 5 6)

                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                      G H

                                      3 5 6 3 4 5

                                      36

                                      Build XFAs from Regex (Cont)

                                      bull Compile to XFAbull Finding efficient implementations

                                      37

                                      Experimental Results

                                      bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                      38

                                      Experimental Results (Cont)

                                      39

                                      Experimental Results (Cont)

                                      40

                                      Conclusion

                                      bull DFAs for regular expressions often blow up when combined

                                      bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                      bull Result compared to other feasible approaches reduce both time and space

                                      41

                                      • XFA Faster Signature Matching With Extended Automata
                                      • Outline
                                      • Introduction
                                      • Technical overview
                                      • Technical overview (Cont)
                                      • Technical overview (Cont) (2)
                                      • Technical overview (Cont) (3)
                                      • Technical overview (Cont) (4)
                                      • Build XFAs from Regex
                                      • Build XFAs from Regex (Cont)
                                      • Build XFAs from Regex (Cont) (2)
                                      • Build XFAs from Regex (Cont) (3)
                                      • Build XFAs from Regex (Cont) (4)
                                      • Build XFAs from Regex (Cont) (5)
                                      • Build XFAs from Regex (Cont) (6)
                                      • Build XFAs from Regex (Cont) (7)
                                      • Build XFAs from Regex (Cont) (8)
                                      • Build XFAs from Regex (Cont) (9)
                                      • Build XFAs from Regex (Cont) (10)
                                      • Build XFAs from Regex (Cont) (11)
                                      • Build XFAs from Regex (Cont) (12)
                                      • Build XFAs from Regex (Cont) (13)
                                      • Build XFAs from Regex (Cont) (14)
                                      • Build XFAs from Regex (Cont) (15)
                                      • Build XFAs from Regex (Cont) (16)
                                      • Build XFAs from Regex (Cont) (17)
                                      • Build XFAs from Regex (Cont) (18)
                                      • Build XFAs from Regex (Cont) (19)
                                      • Build XFAs from Regex (Cont) (20)
                                      • Build XFAs from Regex (Cont) (21)
                                      • Build XFAs from Regex (Cont) (22)
                                      • Build XFAs from Regex (Cont) (23)
                                      • Build XFAs from Regex (Cont) (24)
                                      • Build XFAs from Regex (Cont) (25)
                                      • Build XFAs from Regex (Cont) (26)
                                      • Build XFAs from Regex (Cont) (27)
                                      • Build XFAs from Regex (Cont) (28)
                                      • Experimental Results
                                      • Experimental Results (Cont)
                                      • Experimental Results (Cont) (2)
                                      • Conclusion

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                        Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                        Drsquo (A 0) (B 1) (C 0) (C 2)

                                        δlsquo A C A C

                                        A C

                                        Frsquo

                                        A C

                                        A C

                                        20

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                                        δlsquo A C A C A C A B C

                                        A C

                                        Frsquo

                                        A B C

                                        A C

                                        21

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                                        δlsquo A C A C A C A B C A B C A B C

                                        A C

                                        Frsquo

                                        A B C

                                        A B C

                                        22

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                        Drsquo (A 0) (B 0) (C 0) (C 2)

                                        δlsquo A C A C A C A B C A B C A B C A B C A C

                                        A C

                                        Frsquo

                                        A C

                                        A B C

                                        23

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                        Drsquo (A 0) (B 0) (C 0) (C 2)

                                        δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                        A C

                                        Frsquo

                                        A C

                                        A B C

                                        24

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                        Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                        Drsquo (A 0) (B 0) (C 0) (C 2)

                                        δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                        A C

                                        Frsquo (A C (C 1)) (A B C (C 1))

                                        A C

                                        A B C

                                        25

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H

                                        Drsquo 3 5

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5)

                                        26

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5

                                        Drsquo 3 5

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5)

                                        G G

                                        35 35

                                        27

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5)

                                        G H

                                        35 3 4 5

                                        28

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5)

                                        H H

                                        3 4 5 3 4 5

                                        29

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                        Drsquo 3 5 3 4 5 3 5 7

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                        H G

                                        3 4 5 3 5 7

                                        30

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                        Drsquo 3 5 3 4 5 3 5 7

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                        H G

                                        3 4 5 3 5

                                        31

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                        G G

                                        3 5 7 3 5 6

                                        32

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                        G H

                                        3 5 7 3 4 5

                                        33

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                        G G

                                        3 5 6 3 5

                                        34

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo

                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                        G H

                                        3 5 6 3 4 5

                                        35

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                        δlsquo G G G H H H H G H G

                                        ( ) (G 3 5)

                                        Frsquo (G 3 5 6)

                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                        G H

                                        3 5 6 3 4 5

                                        36

                                        Build XFAs from Regex (Cont)

                                        bull Compile to XFAbull Finding efficient implementations

                                        37

                                        Experimental Results

                                        bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                        38

                                        Experimental Results (Cont)

                                        39

                                        Experimental Results (Cont)

                                        40

                                        Conclusion

                                        bull DFAs for regular expressions often blow up when combined

                                        bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                        bull Result compared to other feasible approaches reduce both time and space

                                        41

                                        • XFA Faster Signature Matching With Extended Automata
                                        • Outline
                                        • Introduction
                                        • Technical overview
                                        • Technical overview (Cont)
                                        • Technical overview (Cont) (2)
                                        • Technical overview (Cont) (3)
                                        • Technical overview (Cont) (4)
                                        • Build XFAs from Regex
                                        • Build XFAs from Regex (Cont)
                                        • Build XFAs from Regex (Cont) (2)
                                        • Build XFAs from Regex (Cont) (3)
                                        • Build XFAs from Regex (Cont) (4)
                                        • Build XFAs from Regex (Cont) (5)
                                        • Build XFAs from Regex (Cont) (6)
                                        • Build XFAs from Regex (Cont) (7)
                                        • Build XFAs from Regex (Cont) (8)
                                        • Build XFAs from Regex (Cont) (9)
                                        • Build XFAs from Regex (Cont) (10)
                                        • Build XFAs from Regex (Cont) (11)
                                        • Build XFAs from Regex (Cont) (12)
                                        • Build XFAs from Regex (Cont) (13)
                                        • Build XFAs from Regex (Cont) (14)
                                        • Build XFAs from Regex (Cont) (15)
                                        • Build XFAs from Regex (Cont) (16)
                                        • Build XFAs from Regex (Cont) (17)
                                        • Build XFAs from Regex (Cont) (18)
                                        • Build XFAs from Regex (Cont) (19)
                                        • Build XFAs from Regex (Cont) (20)
                                        • Build XFAs from Regex (Cont) (21)
                                        • Build XFAs from Regex (Cont) (22)
                                        • Build XFAs from Regex (Cont) (23)
                                        • Build XFAs from Regex (Cont) (24)
                                        • Build XFAs from Regex (Cont) (25)
                                        • Build XFAs from Regex (Cont) (26)
                                        • Build XFAs from Regex (Cont) (27)
                                        • Build XFAs from Regex (Cont) (28)
                                        • Experimental Results
                                        • Experimental Results (Cont)
                                        • Experimental Results (Cont) (2)
                                        • Conclusion

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

                                          δlsquo A C A C A C A B C

                                          A C

                                          Frsquo

                                          A B C

                                          A C

                                          21

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                                          δlsquo A C A C A C A B C A B C A B C

                                          A C

                                          Frsquo

                                          A B C

                                          A B C

                                          22

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                          Drsquo (A 0) (B 0) (C 0) (C 2)

                                          δlsquo A C A C A C A B C A B C A B C A B C A C

                                          A C

                                          Frsquo

                                          A C

                                          A B C

                                          23

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                          Drsquo (A 0) (B 0) (C 0) (C 2)

                                          δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                          A C

                                          Frsquo

                                          A C

                                          A B C

                                          24

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                          Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                          Drsquo (A 0) (B 0) (C 0) (C 2)

                                          δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                          A C

                                          Frsquo (A C (C 1)) (A B C (C 1))

                                          A C

                                          A B C

                                          25

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H

                                          Drsquo 3 5

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5)

                                          26

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5

                                          Drsquo 3 5

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5)

                                          G G

                                          35 35

                                          27

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5)

                                          G H

                                          35 3 4 5

                                          28

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5)

                                          H H

                                          3 4 5 3 4 5

                                          29

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                          Drsquo 3 5 3 4 5 3 5 7

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                          H G

                                          3 4 5 3 5 7

                                          30

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                          Drsquo 3 5 3 4 5 3 5 7

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                          H G

                                          3 4 5 3 5

                                          31

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                          G G

                                          3 5 7 3 5 6

                                          32

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                          G H

                                          3 5 7 3 4 5

                                          33

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                          G G

                                          3 5 6 3 5

                                          34

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo

                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                          G H

                                          3 5 6 3 4 5

                                          35

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                          δlsquo G G G H H H H G H G

                                          ( ) (G 3 5)

                                          Frsquo (G 3 5 6)

                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                          G H

                                          3 5 6 3 4 5

                                          36

                                          Build XFAs from Regex (Cont)

                                          bull Compile to XFAbull Finding efficient implementations

                                          37

                                          Experimental Results

                                          bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                          38

                                          Experimental Results (Cont)

                                          39

                                          Experimental Results (Cont)

                                          40

                                          Conclusion

                                          bull DFAs for regular expressions often blow up when combined

                                          bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                          bull Result compared to other feasible approaches reduce both time and space

                                          41

                                          • XFA Faster Signature Matching With Extended Automata
                                          • Outline
                                          • Introduction
                                          • Technical overview
                                          • Technical overview (Cont)
                                          • Technical overview (Cont) (2)
                                          • Technical overview (Cont) (3)
                                          • Technical overview (Cont) (4)
                                          • Build XFAs from Regex
                                          • Build XFAs from Regex (Cont)
                                          • Build XFAs from Regex (Cont) (2)
                                          • Build XFAs from Regex (Cont) (3)
                                          • Build XFAs from Regex (Cont) (4)
                                          • Build XFAs from Regex (Cont) (5)
                                          • Build XFAs from Regex (Cont) (6)
                                          • Build XFAs from Regex (Cont) (7)
                                          • Build XFAs from Regex (Cont) (8)
                                          • Build XFAs from Regex (Cont) (9)
                                          • Build XFAs from Regex (Cont) (10)
                                          • Build XFAs from Regex (Cont) (11)
                                          • Build XFAs from Regex (Cont) (12)
                                          • Build XFAs from Regex (Cont) (13)
                                          • Build XFAs from Regex (Cont) (14)
                                          • Build XFAs from Regex (Cont) (15)
                                          • Build XFAs from Regex (Cont) (16)
                                          • Build XFAs from Regex (Cont) (17)
                                          • Build XFAs from Regex (Cont) (18)
                                          • Build XFAs from Regex (Cont) (19)
                                          • Build XFAs from Regex (Cont) (20)
                                          • Build XFAs from Regex (Cont) (21)
                                          • Build XFAs from Regex (Cont) (22)
                                          • Build XFAs from Regex (Cont) (23)
                                          • Build XFAs from Regex (Cont) (24)
                                          • Build XFAs from Regex (Cont) (25)
                                          • Build XFAs from Regex (Cont) (26)
                                          • Build XFAs from Regex (Cont) (27)
                                          • Build XFAs from Regex (Cont) (28)
                                          • Experimental Results
                                          • Experimental Results (Cont)
                                          • Experimental Results (Cont) (2)
                                          • Conclusion

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

                                            δlsquo A C A C A C A B C A B C A B C

                                            A C

                                            Frsquo

                                            A B C

                                            A B C

                                            22

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                            Drsquo (A 0) (B 0) (C 0) (C 2)

                                            δlsquo A C A C A C A B C A B C A B C A B C A C

                                            A C

                                            Frsquo

                                            A C

                                            A B C

                                            23

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                            Drsquo (A 0) (B 0) (C 0) (C 2)

                                            δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                            A C

                                            Frsquo

                                            A C

                                            A B C

                                            24

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                            Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                            Drsquo (A 0) (B 0) (C 0) (C 2)

                                            δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                            A C

                                            Frsquo (A C (C 1)) (A B C (C 1))

                                            A C

                                            A B C

                                            25

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H

                                            Drsquo 3 5

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5)

                                            26

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5

                                            Drsquo 3 5

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5)

                                            G G

                                            35 35

                                            27

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5)

                                            G H

                                            35 3 4 5

                                            28

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5)

                                            H H

                                            3 4 5 3 4 5

                                            29

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                            Drsquo 3 5 3 4 5 3 5 7

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                            H G

                                            3 4 5 3 5 7

                                            30

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                            Drsquo 3 5 3 4 5 3 5 7

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                            H G

                                            3 4 5 3 5

                                            31

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                            G G

                                            3 5 7 3 5 6

                                            32

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                            G H

                                            3 5 7 3 4 5

                                            33

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                            G G

                                            3 5 6 3 5

                                            34

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo

                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                            G H

                                            3 5 6 3 4 5

                                            35

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                            δlsquo G G G H H H H G H G

                                            ( ) (G 3 5)

                                            Frsquo (G 3 5 6)

                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                            G H

                                            3 5 6 3 4 5

                                            36

                                            Build XFAs from Regex (Cont)

                                            bull Compile to XFAbull Finding efficient implementations

                                            37

                                            Experimental Results

                                            bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                            38

                                            Experimental Results (Cont)

                                            39

                                            Experimental Results (Cont)

                                            40

                                            Conclusion

                                            bull DFAs for regular expressions often blow up when combined

                                            bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                            bull Result compared to other feasible approaches reduce both time and space

                                            41

                                            • XFA Faster Signature Matching With Extended Automata
                                            • Outline
                                            • Introduction
                                            • Technical overview
                                            • Technical overview (Cont)
                                            • Technical overview (Cont) (2)
                                            • Technical overview (Cont) (3)
                                            • Technical overview (Cont) (4)
                                            • Build XFAs from Regex
                                            • Build XFAs from Regex (Cont)
                                            • Build XFAs from Regex (Cont) (2)
                                            • Build XFAs from Regex (Cont) (3)
                                            • Build XFAs from Regex (Cont) (4)
                                            • Build XFAs from Regex (Cont) (5)
                                            • Build XFAs from Regex (Cont) (6)
                                            • Build XFAs from Regex (Cont) (7)
                                            • Build XFAs from Regex (Cont) (8)
                                            • Build XFAs from Regex (Cont) (9)
                                            • Build XFAs from Regex (Cont) (10)
                                            • Build XFAs from Regex (Cont) (11)
                                            • Build XFAs from Regex (Cont) (12)
                                            • Build XFAs from Regex (Cont) (13)
                                            • Build XFAs from Regex (Cont) (14)
                                            • Build XFAs from Regex (Cont) (15)
                                            • Build XFAs from Regex (Cont) (16)
                                            • Build XFAs from Regex (Cont) (17)
                                            • Build XFAs from Regex (Cont) (18)
                                            • Build XFAs from Regex (Cont) (19)
                                            • Build XFAs from Regex (Cont) (20)
                                            • Build XFAs from Regex (Cont) (21)
                                            • Build XFAs from Regex (Cont) (22)
                                            • Build XFAs from Regex (Cont) (23)
                                            • Build XFAs from Regex (Cont) (24)
                                            • Build XFAs from Regex (Cont) (25)
                                            • Build XFAs from Regex (Cont) (26)
                                            • Build XFAs from Regex (Cont) (27)
                                            • Build XFAs from Regex (Cont) (28)
                                            • Experimental Results
                                            • Experimental Results (Cont)
                                            • Experimental Results (Cont) (2)
                                            • Conclusion

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

                                              Drsquo (A 0) (B 0) (C 0) (C 2)

                                              δlsquo A C A C A C A B C A B C A B C A B C A C

                                              A C

                                              Frsquo

                                              A C

                                              A B C

                                              23

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                              Drsquo (A 0) (B 0) (C 0) (C 2)

                                              δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                              A C

                                              Frsquo

                                              A C

                                              A B C

                                              24

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                              Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                              Drsquo (A 0) (B 0) (C 0) (C 2)

                                              δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                              A C

                                              Frsquo (A C (C 1)) (A B C (C 1))

                                              A C

                                              A B C

                                              25

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H

                                              Drsquo 3 5

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5)

                                              26

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5

                                              Drsquo 3 5

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5)

                                              G G

                                              35 35

                                              27

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5)

                                              G H

                                              35 3 4 5

                                              28

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5)

                                              H H

                                              3 4 5 3 4 5

                                              29

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                              Drsquo 3 5 3 4 5 3 5 7

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                              H G

                                              3 4 5 3 5 7

                                              30

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                              Drsquo 3 5 3 4 5 3 5 7

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                              H G

                                              3 4 5 3 5

                                              31

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                              G G

                                              3 5 7 3 5 6

                                              32

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                              G H

                                              3 5 7 3 4 5

                                              33

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                              G G

                                              3 5 6 3 5

                                              34

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo

                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                              G H

                                              3 5 6 3 4 5

                                              35

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                              δlsquo G G G H H H H G H G

                                              ( ) (G 3 5)

                                              Frsquo (G 3 5 6)

                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                              G H

                                              3 5 6 3 4 5

                                              36

                                              Build XFAs from Regex (Cont)

                                              bull Compile to XFAbull Finding efficient implementations

                                              37

                                              Experimental Results

                                              bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                              38

                                              Experimental Results (Cont)

                                              39

                                              Experimental Results (Cont)

                                              40

                                              Conclusion

                                              bull DFAs for regular expressions often blow up when combined

                                              bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                              bull Result compared to other feasible approaches reduce both time and space

                                              41

                                              • XFA Faster Signature Matching With Extended Automata
                                              • Outline
                                              • Introduction
                                              • Technical overview
                                              • Technical overview (Cont)
                                              • Technical overview (Cont) (2)
                                              • Technical overview (Cont) (3)
                                              • Technical overview (Cont) (4)
                                              • Build XFAs from Regex
                                              • Build XFAs from Regex (Cont)
                                              • Build XFAs from Regex (Cont) (2)
                                              • Build XFAs from Regex (Cont) (3)
                                              • Build XFAs from Regex (Cont) (4)
                                              • Build XFAs from Regex (Cont) (5)
                                              • Build XFAs from Regex (Cont) (6)
                                              • Build XFAs from Regex (Cont) (7)
                                              • Build XFAs from Regex (Cont) (8)
                                              • Build XFAs from Regex (Cont) (9)
                                              • Build XFAs from Regex (Cont) (10)
                                              • Build XFAs from Regex (Cont) (11)
                                              • Build XFAs from Regex (Cont) (12)
                                              • Build XFAs from Regex (Cont) (13)
                                              • Build XFAs from Regex (Cont) (14)
                                              • Build XFAs from Regex (Cont) (15)
                                              • Build XFAs from Regex (Cont) (16)
                                              • Build XFAs from Regex (Cont) (17)
                                              • Build XFAs from Regex (Cont) (18)
                                              • Build XFAs from Regex (Cont) (19)
                                              • Build XFAs from Regex (Cont) (20)
                                              • Build XFAs from Regex (Cont) (21)
                                              • Build XFAs from Regex (Cont) (22)
                                              • Build XFAs from Regex (Cont) (23)
                                              • Build XFAs from Regex (Cont) (24)
                                              • Build XFAs from Regex (Cont) (25)
                                              • Build XFAs from Regex (Cont) (26)
                                              • Build XFAs from Regex (Cont) (27)
                                              • Build XFAs from Regex (Cont) (28)
                                              • Experimental Results
                                              • Experimental Results (Cont)
                                              • Experimental Results (Cont) (2)
                                              • Conclusion

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                                Drsquo (A 0) (B 0) (C 0) (C 2)

                                                δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                                A C

                                                Frsquo

                                                A C

                                                A B C

                                                24

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                                Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                                Drsquo (A 0) (B 0) (C 0) (C 2)

                                                δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                                A C

                                                Frsquo (A C (C 1)) (A B C (C 1))

                                                A C

                                                A B C

                                                25

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H

                                                Drsquo 3 5

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5)

                                                26

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5

                                                Drsquo 3 5

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5)

                                                G G

                                                35 35

                                                27

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5)

                                                G H

                                                35 3 4 5

                                                28

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5)

                                                H H

                                                3 4 5 3 4 5

                                                29

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                                Drsquo 3 5 3 4 5 3 5 7

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                H G

                                                3 4 5 3 5 7

                                                30

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                Drsquo 3 5 3 4 5 3 5 7

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                H G

                                                3 4 5 3 5

                                                31

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                G G

                                                3 5 7 3 5 6

                                                32

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                G H

                                                3 5 7 3 4 5

                                                33

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                G G

                                                3 5 6 3 5

                                                34

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo

                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                G H

                                                3 5 6 3 4 5

                                                35

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                δlsquo G G G H H H H G H G

                                                ( ) (G 3 5)

                                                Frsquo (G 3 5 6)

                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                G H

                                                3 5 6 3 4 5

                                                36

                                                Build XFAs from Regex (Cont)

                                                bull Compile to XFAbull Finding efficient implementations

                                                37

                                                Experimental Results

                                                bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                38

                                                Experimental Results (Cont)

                                                39

                                                Experimental Results (Cont)

                                                40

                                                Conclusion

                                                bull DFAs for regular expressions often blow up when combined

                                                bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                bull Result compared to other feasible approaches reduce both time and space

                                                41

                                                • XFA Faster Signature Matching With Extended Automata
                                                • Outline
                                                • Introduction
                                                • Technical overview
                                                • Technical overview (Cont)
                                                • Technical overview (Cont) (2)
                                                • Technical overview (Cont) (3)
                                                • Technical overview (Cont) (4)
                                                • Build XFAs from Regex
                                                • Build XFAs from Regex (Cont)
                                                • Build XFAs from Regex (Cont) (2)
                                                • Build XFAs from Regex (Cont) (3)
                                                • Build XFAs from Regex (Cont) (4)
                                                • Build XFAs from Regex (Cont) (5)
                                                • Build XFAs from Regex (Cont) (6)
                                                • Build XFAs from Regex (Cont) (7)
                                                • Build XFAs from Regex (Cont) (8)
                                                • Build XFAs from Regex (Cont) (9)
                                                • Build XFAs from Regex (Cont) (10)
                                                • Build XFAs from Regex (Cont) (11)
                                                • Build XFAs from Regex (Cont) (12)
                                                • Build XFAs from Regex (Cont) (13)
                                                • Build XFAs from Regex (Cont) (14)
                                                • Build XFAs from Regex (Cont) (15)
                                                • Build XFAs from Regex (Cont) (16)
                                                • Build XFAs from Regex (Cont) (17)
                                                • Build XFAs from Regex (Cont) (18)
                                                • Build XFAs from Regex (Cont) (19)
                                                • Build XFAs from Regex (Cont) (20)
                                                • Build XFAs from Regex (Cont) (21)
                                                • Build XFAs from Regex (Cont) (22)
                                                • Build XFAs from Regex (Cont) (23)
                                                • Build XFAs from Regex (Cont) (24)
                                                • Build XFAs from Regex (Cont) (25)
                                                • Build XFAs from Regex (Cont) (26)
                                                • Build XFAs from Regex (Cont) (27)
                                                • Build XFAs from Regex (Cont) (28)
                                                • Experimental Results
                                                • Experimental Results (Cont)
                                                • Experimental Results (Cont) (2)
                                                • Conclusion

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

                                                  Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

                                                  Drsquo (A 0) (B 0) (C 0) (C 2)

                                                  δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

                                                  A C

                                                  Frsquo (A C (C 1)) (A B C (C 1))

                                                  A C

                                                  A B C

                                                  25

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H

                                                  Drsquo 3 5

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5)

                                                  26

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5

                                                  Drsquo 3 5

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5)

                                                  G G

                                                  35 35

                                                  27

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5)

                                                  G H

                                                  35 3 4 5

                                                  28

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5)

                                                  H H

                                                  3 4 5 3 4 5

                                                  29

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                                  Drsquo 3 5 3 4 5 3 5 7

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                  H G

                                                  3 4 5 3 5 7

                                                  30

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                  Drsquo 3 5 3 4 5 3 5 7

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                  H G

                                                  3 4 5 3 5

                                                  31

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                  G G

                                                  3 5 7 3 5 6

                                                  32

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                  G H

                                                  3 5 7 3 4 5

                                                  33

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                  G G

                                                  3 5 6 3 5

                                                  34

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo

                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                  G H

                                                  3 5 6 3 4 5

                                                  35

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                  δlsquo G G G H H H H G H G

                                                  ( ) (G 3 5)

                                                  Frsquo (G 3 5 6)

                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                  G H

                                                  3 5 6 3 4 5

                                                  36

                                                  Build XFAs from Regex (Cont)

                                                  bull Compile to XFAbull Finding efficient implementations

                                                  37

                                                  Experimental Results

                                                  bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                  38

                                                  Experimental Results (Cont)

                                                  39

                                                  Experimental Results (Cont)

                                                  40

                                                  Conclusion

                                                  bull DFAs for regular expressions often blow up when combined

                                                  bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                  bull Result compared to other feasible approaches reduce both time and space

                                                  41

                                                  • XFA Faster Signature Matching With Extended Automata
                                                  • Outline
                                                  • Introduction
                                                  • Technical overview
                                                  • Technical overview (Cont)
                                                  • Technical overview (Cont) (2)
                                                  • Technical overview (Cont) (3)
                                                  • Technical overview (Cont) (4)
                                                  • Build XFAs from Regex
                                                  • Build XFAs from Regex (Cont)
                                                  • Build XFAs from Regex (Cont) (2)
                                                  • Build XFAs from Regex (Cont) (3)
                                                  • Build XFAs from Regex (Cont) (4)
                                                  • Build XFAs from Regex (Cont) (5)
                                                  • Build XFAs from Regex (Cont) (6)
                                                  • Build XFAs from Regex (Cont) (7)
                                                  • Build XFAs from Regex (Cont) (8)
                                                  • Build XFAs from Regex (Cont) (9)
                                                  • Build XFAs from Regex (Cont) (10)
                                                  • Build XFAs from Regex (Cont) (11)
                                                  • Build XFAs from Regex (Cont) (12)
                                                  • Build XFAs from Regex (Cont) (13)
                                                  • Build XFAs from Regex (Cont) (14)
                                                  • Build XFAs from Regex (Cont) (15)
                                                  • Build XFAs from Regex (Cont) (16)
                                                  • Build XFAs from Regex (Cont) (17)
                                                  • Build XFAs from Regex (Cont) (18)
                                                  • Build XFAs from Regex (Cont) (19)
                                                  • Build XFAs from Regex (Cont) (20)
                                                  • Build XFAs from Regex (Cont) (21)
                                                  • Build XFAs from Regex (Cont) (22)
                                                  • Build XFAs from Regex (Cont) (23)
                                                  • Build XFAs from Regex (Cont) (24)
                                                  • Build XFAs from Regex (Cont) (25)
                                                  • Build XFAs from Regex (Cont) (26)
                                                  • Build XFAs from Regex (Cont) (27)
                                                  • Build XFAs from Regex (Cont) (28)
                                                  • Experimental Results
                                                  • Experimental Results (Cont)
                                                  • Experimental Results (Cont) (2)
                                                  • Conclusion

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H

                                                    Drsquo 3 5

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5)

                                                    26

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5

                                                    Drsquo 3 5

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5)

                                                    G G

                                                    35 35

                                                    27

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5)

                                                    G H

                                                    35 3 4 5

                                                    28

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5)

                                                    H H

                                                    3 4 5 3 4 5

                                                    29

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                                    Drsquo 3 5 3 4 5 3 5 7

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                    H G

                                                    3 4 5 3 5 7

                                                    30

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                    Drsquo 3 5 3 4 5 3 5 7

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                    H G

                                                    3 4 5 3 5

                                                    31

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                    G G

                                                    3 5 7 3 5 6

                                                    32

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                    G H

                                                    3 5 7 3 4 5

                                                    33

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                    G G

                                                    3 5 6 3 5

                                                    34

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo

                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                    G H

                                                    3 5 6 3 4 5

                                                    35

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                    δlsquo G G G H H H H G H G

                                                    ( ) (G 3 5)

                                                    Frsquo (G 3 5 6)

                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                    G H

                                                    3 5 6 3 4 5

                                                    36

                                                    Build XFAs from Regex (Cont)

                                                    bull Compile to XFAbull Finding efficient implementations

                                                    37

                                                    Experimental Results

                                                    bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                    38

                                                    Experimental Results (Cont)

                                                    39

                                                    Experimental Results (Cont)

                                                    40

                                                    Conclusion

                                                    bull DFAs for regular expressions often blow up when combined

                                                    bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                    bull Result compared to other feasible approaches reduce both time and space

                                                    41

                                                    • XFA Faster Signature Matching With Extended Automata
                                                    • Outline
                                                    • Introduction
                                                    • Technical overview
                                                    • Technical overview (Cont)
                                                    • Technical overview (Cont) (2)
                                                    • Technical overview (Cont) (3)
                                                    • Technical overview (Cont) (4)
                                                    • Build XFAs from Regex
                                                    • Build XFAs from Regex (Cont)
                                                    • Build XFAs from Regex (Cont) (2)
                                                    • Build XFAs from Regex (Cont) (3)
                                                    • Build XFAs from Regex (Cont) (4)
                                                    • Build XFAs from Regex (Cont) (5)
                                                    • Build XFAs from Regex (Cont) (6)
                                                    • Build XFAs from Regex (Cont) (7)
                                                    • Build XFAs from Regex (Cont) (8)
                                                    • Build XFAs from Regex (Cont) (9)
                                                    • Build XFAs from Regex (Cont) (10)
                                                    • Build XFAs from Regex (Cont) (11)
                                                    • Build XFAs from Regex (Cont) (12)
                                                    • Build XFAs from Regex (Cont) (13)
                                                    • Build XFAs from Regex (Cont) (14)
                                                    • Build XFAs from Regex (Cont) (15)
                                                    • Build XFAs from Regex (Cont) (16)
                                                    • Build XFAs from Regex (Cont) (17)
                                                    • Build XFAs from Regex (Cont) (18)
                                                    • Build XFAs from Regex (Cont) (19)
                                                    • Build XFAs from Regex (Cont) (20)
                                                    • Build XFAs from Regex (Cont) (21)
                                                    • Build XFAs from Regex (Cont) (22)
                                                    • Build XFAs from Regex (Cont) (23)
                                                    • Build XFAs from Regex (Cont) (24)
                                                    • Build XFAs from Regex (Cont) (25)
                                                    • Build XFAs from Regex (Cont) (26)
                                                    • Build XFAs from Regex (Cont) (27)
                                                    • Build XFAs from Regex (Cont) (28)
                                                    • Experimental Results
                                                    • Experimental Results (Cont)
                                                    • Experimental Results (Cont) (2)
                                                    • Conclusion

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5

                                                      Drsquo 3 5

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5)

                                                      G G

                                                      35 35

                                                      27

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5)

                                                      G H

                                                      35 3 4 5

                                                      28

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5)

                                                      H H

                                                      3 4 5 3 4 5

                                                      29

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                                      Drsquo 3 5 3 4 5 3 5 7

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                      H G

                                                      3 4 5 3 5 7

                                                      30

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                      Drsquo 3 5 3 4 5 3 5 7

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                      H G

                                                      3 4 5 3 5

                                                      31

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                      G G

                                                      3 5 7 3 5 6

                                                      32

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                      G H

                                                      3 5 7 3 4 5

                                                      33

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                      G G

                                                      3 5 6 3 5

                                                      34

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo

                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                      G H

                                                      3 5 6 3 4 5

                                                      35

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                      δlsquo G G G H H H H G H G

                                                      ( ) (G 3 5)

                                                      Frsquo (G 3 5 6)

                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                      G H

                                                      3 5 6 3 4 5

                                                      36

                                                      Build XFAs from Regex (Cont)

                                                      bull Compile to XFAbull Finding efficient implementations

                                                      37

                                                      Experimental Results

                                                      bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                      38

                                                      Experimental Results (Cont)

                                                      39

                                                      Experimental Results (Cont)

                                                      40

                                                      Conclusion

                                                      bull DFAs for regular expressions often blow up when combined

                                                      bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                      bull Result compared to other feasible approaches reduce both time and space

                                                      41

                                                      • XFA Faster Signature Matching With Extended Automata
                                                      • Outline
                                                      • Introduction
                                                      • Technical overview
                                                      • Technical overview (Cont)
                                                      • Technical overview (Cont) (2)
                                                      • Technical overview (Cont) (3)
                                                      • Technical overview (Cont) (4)
                                                      • Build XFAs from Regex
                                                      • Build XFAs from Regex (Cont)
                                                      • Build XFAs from Regex (Cont) (2)
                                                      • Build XFAs from Regex (Cont) (3)
                                                      • Build XFAs from Regex (Cont) (4)
                                                      • Build XFAs from Regex (Cont) (5)
                                                      • Build XFAs from Regex (Cont) (6)
                                                      • Build XFAs from Regex (Cont) (7)
                                                      • Build XFAs from Regex (Cont) (8)
                                                      • Build XFAs from Regex (Cont) (9)
                                                      • Build XFAs from Regex (Cont) (10)
                                                      • Build XFAs from Regex (Cont) (11)
                                                      • Build XFAs from Regex (Cont) (12)
                                                      • Build XFAs from Regex (Cont) (13)
                                                      • Build XFAs from Regex (Cont) (14)
                                                      • Build XFAs from Regex (Cont) (15)
                                                      • Build XFAs from Regex (Cont) (16)
                                                      • Build XFAs from Regex (Cont) (17)
                                                      • Build XFAs from Regex (Cont) (18)
                                                      • Build XFAs from Regex (Cont) (19)
                                                      • Build XFAs from Regex (Cont) (20)
                                                      • Build XFAs from Regex (Cont) (21)
                                                      • Build XFAs from Regex (Cont) (22)
                                                      • Build XFAs from Regex (Cont) (23)
                                                      • Build XFAs from Regex (Cont) (24)
                                                      • Build XFAs from Regex (Cont) (25)
                                                      • Build XFAs from Regex (Cont) (26)
                                                      • Build XFAs from Regex (Cont) (27)
                                                      • Build XFAs from Regex (Cont) (28)
                                                      • Experimental Results
                                                      • Experimental Results (Cont)
                                                      • Experimental Results (Cont) (2)
                                                      • Conclusion

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5)

                                                        G H

                                                        35 3 4 5

                                                        28

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5)

                                                        H H

                                                        3 4 5 3 4 5

                                                        29

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                                        Drsquo 3 5 3 4 5 3 5 7

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                        H G

                                                        3 4 5 3 5 7

                                                        30

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                        Drsquo 3 5 3 4 5 3 5 7

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                        H G

                                                        3 4 5 3 5

                                                        31

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                        G G

                                                        3 5 7 3 5 6

                                                        32

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                        G H

                                                        3 5 7 3 4 5

                                                        33

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                        G G

                                                        3 5 6 3 5

                                                        34

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo

                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                        G H

                                                        3 5 6 3 4 5

                                                        35

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                        δlsquo G G G H H H H G H G

                                                        ( ) (G 3 5)

                                                        Frsquo (G 3 5 6)

                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                        G H

                                                        3 5 6 3 4 5

                                                        36

                                                        Build XFAs from Regex (Cont)

                                                        bull Compile to XFAbull Finding efficient implementations

                                                        37

                                                        Experimental Results

                                                        bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                        38

                                                        Experimental Results (Cont)

                                                        39

                                                        Experimental Results (Cont)

                                                        40

                                                        Conclusion

                                                        bull DFAs for regular expressions often blow up when combined

                                                        bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                        bull Result compared to other feasible approaches reduce both time and space

                                                        41

                                                        • XFA Faster Signature Matching With Extended Automata
                                                        • Outline
                                                        • Introduction
                                                        • Technical overview
                                                        • Technical overview (Cont)
                                                        • Technical overview (Cont) (2)
                                                        • Technical overview (Cont) (3)
                                                        • Technical overview (Cont) (4)
                                                        • Build XFAs from Regex
                                                        • Build XFAs from Regex (Cont)
                                                        • Build XFAs from Regex (Cont) (2)
                                                        • Build XFAs from Regex (Cont) (3)
                                                        • Build XFAs from Regex (Cont) (4)
                                                        • Build XFAs from Regex (Cont) (5)
                                                        • Build XFAs from Regex (Cont) (6)
                                                        • Build XFAs from Regex (Cont) (7)
                                                        • Build XFAs from Regex (Cont) (8)
                                                        • Build XFAs from Regex (Cont) (9)
                                                        • Build XFAs from Regex (Cont) (10)
                                                        • Build XFAs from Regex (Cont) (11)
                                                        • Build XFAs from Regex (Cont) (12)
                                                        • Build XFAs from Regex (Cont) (13)
                                                        • Build XFAs from Regex (Cont) (14)
                                                        • Build XFAs from Regex (Cont) (15)
                                                        • Build XFAs from Regex (Cont) (16)
                                                        • Build XFAs from Regex (Cont) (17)
                                                        • Build XFAs from Regex (Cont) (18)
                                                        • Build XFAs from Regex (Cont) (19)
                                                        • Build XFAs from Regex (Cont) (20)
                                                        • Build XFAs from Regex (Cont) (21)
                                                        • Build XFAs from Regex (Cont) (22)
                                                        • Build XFAs from Regex (Cont) (23)
                                                        • Build XFAs from Regex (Cont) (24)
                                                        • Build XFAs from Regex (Cont) (25)
                                                        • Build XFAs from Regex (Cont) (26)
                                                        • Build XFAs from Regex (Cont) (27)
                                                        • Build XFAs from Regex (Cont) (28)
                                                        • Experimental Results
                                                        • Experimental Results (Cont)
                                                        • Experimental Results (Cont) (2)
                                                        • Conclusion

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo

                                                          QD (G 3 5) (H 3 4 5)

                                                          H H

                                                          3 4 5 3 4 5

                                                          29

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                                          Drsquo 3 5 3 4 5 3 5 7

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo

                                                          QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                          H G

                                                          3 4 5 3 5 7

                                                          30

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                          Drsquo 3 5 3 4 5 3 5 7

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo

                                                          QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                          H G

                                                          3 4 5 3 5

                                                          31

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo

                                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                          G G

                                                          3 5 7 3 5 6

                                                          32

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo

                                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                          G H

                                                          3 5 7 3 4 5

                                                          33

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo

                                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                          G G

                                                          3 5 6 3 5

                                                          34

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo

                                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                          G H

                                                          3 5 6 3 4 5

                                                          35

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                          Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                          Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                          δlsquo G G G H H H H G H G

                                                          ( ) (G 3 5)

                                                          Frsquo (G 3 5 6)

                                                          QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                          G H

                                                          3 5 6 3 4 5

                                                          36

                                                          Build XFAs from Regex (Cont)

                                                          bull Compile to XFAbull Finding efficient implementations

                                                          37

                                                          Experimental Results

                                                          bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                          38

                                                          Experimental Results (Cont)

                                                          39

                                                          Experimental Results (Cont)

                                                          40

                                                          Conclusion

                                                          bull DFAs for regular expressions often blow up when combined

                                                          bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                          bull Result compared to other feasible approaches reduce both time and space

                                                          41

                                                          • XFA Faster Signature Matching With Extended Automata
                                                          • Outline
                                                          • Introduction
                                                          • Technical overview
                                                          • Technical overview (Cont)
                                                          • Technical overview (Cont) (2)
                                                          • Technical overview (Cont) (3)
                                                          • Technical overview (Cont) (4)
                                                          • Build XFAs from Regex
                                                          • Build XFAs from Regex (Cont)
                                                          • Build XFAs from Regex (Cont) (2)
                                                          • Build XFAs from Regex (Cont) (3)
                                                          • Build XFAs from Regex (Cont) (4)
                                                          • Build XFAs from Regex (Cont) (5)
                                                          • Build XFAs from Regex (Cont) (6)
                                                          • Build XFAs from Regex (Cont) (7)
                                                          • Build XFAs from Regex (Cont) (8)
                                                          • Build XFAs from Regex (Cont) (9)
                                                          • Build XFAs from Regex (Cont) (10)
                                                          • Build XFAs from Regex (Cont) (11)
                                                          • Build XFAs from Regex (Cont) (12)
                                                          • Build XFAs from Regex (Cont) (13)
                                                          • Build XFAs from Regex (Cont) (14)
                                                          • Build XFAs from Regex (Cont) (15)
                                                          • Build XFAs from Regex (Cont) (16)
                                                          • Build XFAs from Regex (Cont) (17)
                                                          • Build XFAs from Regex (Cont) (18)
                                                          • Build XFAs from Regex (Cont) (19)
                                                          • Build XFAs from Regex (Cont) (20)
                                                          • Build XFAs from Regex (Cont) (21)
                                                          • Build XFAs from Regex (Cont) (22)
                                                          • Build XFAs from Regex (Cont) (23)
                                                          • Build XFAs from Regex (Cont) (24)
                                                          • Build XFAs from Regex (Cont) (25)
                                                          • Build XFAs from Regex (Cont) (26)
                                                          • Build XFAs from Regex (Cont) (27)
                                                          • Build XFAs from Regex (Cont) (28)
                                                          • Experimental Results
                                                          • Experimental Results (Cont)
                                                          • Experimental Results (Cont) (2)
                                                          • Conclusion

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

                                                            Drsquo 3 5 3 4 5 3 5 7

                                                            δlsquo G G G H H H H G H G

                                                            ( ) (G 3 5)

                                                            Frsquo

                                                            QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                            H G

                                                            3 4 5 3 5 7

                                                            30

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                            Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                            Drsquo 3 5 3 4 5 3 5 7

                                                            δlsquo G G G H H H H G H G

                                                            ( ) (G 3 5)

                                                            Frsquo

                                                            QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                            H G

                                                            3 4 5 3 5

                                                            31

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                            δlsquo G G G H H H H G H G

                                                            ( ) (G 3 5)

                                                            Frsquo

                                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                            G G

                                                            3 5 7 3 5 6

                                                            32

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                            δlsquo G G G H H H H G H G

                                                            ( ) (G 3 5)

                                                            Frsquo

                                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                            G H

                                                            3 5 7 3 4 5

                                                            33

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                            δlsquo G G G H H H H G H G

                                                            ( ) (G 3 5)

                                                            Frsquo

                                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                            G G

                                                            3 5 6 3 5

                                                            34

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                            δlsquo G G G H H H H G H G

                                                            ( ) (G 3 5)

                                                            Frsquo

                                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                            G H

                                                            3 5 6 3 4 5

                                                            35

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                            Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                            Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                            δlsquo G G G H H H H G H G

                                                            ( ) (G 3 5)

                                                            Frsquo (G 3 5 6)

                                                            QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                            G H

                                                            3 5 6 3 4 5

                                                            36

                                                            Build XFAs from Regex (Cont)

                                                            bull Compile to XFAbull Finding efficient implementations

                                                            37

                                                            Experimental Results

                                                            bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                            38

                                                            Experimental Results (Cont)

                                                            39

                                                            Experimental Results (Cont)

                                                            40

                                                            Conclusion

                                                            bull DFAs for regular expressions often blow up when combined

                                                            bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                            bull Result compared to other feasible approaches reduce both time and space

                                                            41

                                                            • XFA Faster Signature Matching With Extended Automata
                                                            • Outline
                                                            • Introduction
                                                            • Technical overview
                                                            • Technical overview (Cont)
                                                            • Technical overview (Cont) (2)
                                                            • Technical overview (Cont) (3)
                                                            • Technical overview (Cont) (4)
                                                            • Build XFAs from Regex
                                                            • Build XFAs from Regex (Cont)
                                                            • Build XFAs from Regex (Cont) (2)
                                                            • Build XFAs from Regex (Cont) (3)
                                                            • Build XFAs from Regex (Cont) (4)
                                                            • Build XFAs from Regex (Cont) (5)
                                                            • Build XFAs from Regex (Cont) (6)
                                                            • Build XFAs from Regex (Cont) (7)
                                                            • Build XFAs from Regex (Cont) (8)
                                                            • Build XFAs from Regex (Cont) (9)
                                                            • Build XFAs from Regex (Cont) (10)
                                                            • Build XFAs from Regex (Cont) (11)
                                                            • Build XFAs from Regex (Cont) (12)
                                                            • Build XFAs from Regex (Cont) (13)
                                                            • Build XFAs from Regex (Cont) (14)
                                                            • Build XFAs from Regex (Cont) (15)
                                                            • Build XFAs from Regex (Cont) (16)
                                                            • Build XFAs from Regex (Cont) (17)
                                                            • Build XFAs from Regex (Cont) (18)
                                                            • Build XFAs from Regex (Cont) (19)
                                                            • Build XFAs from Regex (Cont) (20)
                                                            • Build XFAs from Regex (Cont) (21)
                                                            • Build XFAs from Regex (Cont) (22)
                                                            • Build XFAs from Regex (Cont) (23)
                                                            • Build XFAs from Regex (Cont) (24)
                                                            • Build XFAs from Regex (Cont) (25)
                                                            • Build XFAs from Regex (Cont) (26)
                                                            • Build XFAs from Regex (Cont) (27)
                                                            • Build XFAs from Regex (Cont) (28)
                                                            • Experimental Results
                                                            • Experimental Results (Cont)
                                                            • Experimental Results (Cont) (2)
                                                            • Conclusion

                                                              Build XFAs from Regex (Cont)

                                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                              Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                              Drsquo 3 5 3 4 5 3 5 7

                                                              δlsquo G G G H H H H G H G

                                                              ( ) (G 3 5)

                                                              Frsquo

                                                              QD (G 3 5) (H 3 4 5) (G 3 5 7)

                                                              H G

                                                              3 4 5 3 5

                                                              31

                                                              Build XFAs from Regex (Cont)

                                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                              δlsquo G G G H H H H G H G

                                                              ( ) (G 3 5)

                                                              Frsquo

                                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                              G G

                                                              3 5 7 3 5 6

                                                              32

                                                              Build XFAs from Regex (Cont)

                                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                              δlsquo G G G H H H H G H G

                                                              ( ) (G 3 5)

                                                              Frsquo

                                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                              G H

                                                              3 5 7 3 4 5

                                                              33

                                                              Build XFAs from Regex (Cont)

                                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                              δlsquo G G G H H H H G H G

                                                              ( ) (G 3 5)

                                                              Frsquo

                                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                              G G

                                                              3 5 6 3 5

                                                              34

                                                              Build XFAs from Regex (Cont)

                                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                              δlsquo G G G H H H H G H G

                                                              ( ) (G 3 5)

                                                              Frsquo

                                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                              G H

                                                              3 5 6 3 4 5

                                                              35

                                                              Build XFAs from Regex (Cont)

                                                              bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                              Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                              Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                              δlsquo G G G H H H H G H G

                                                              ( ) (G 3 5)

                                                              Frsquo (G 3 5 6)

                                                              QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                              G H

                                                              3 5 6 3 4 5

                                                              36

                                                              Build XFAs from Regex (Cont)

                                                              bull Compile to XFAbull Finding efficient implementations

                                                              37

                                                              Experimental Results

                                                              bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                              38

                                                              Experimental Results (Cont)

                                                              39

                                                              Experimental Results (Cont)

                                                              40

                                                              Conclusion

                                                              bull DFAs for regular expressions often blow up when combined

                                                              bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                              bull Result compared to other feasible approaches reduce both time and space

                                                              41

                                                              • XFA Faster Signature Matching With Extended Automata
                                                              • Outline
                                                              • Introduction
                                                              • Technical overview
                                                              • Technical overview (Cont)
                                                              • Technical overview (Cont) (2)
                                                              • Technical overview (Cont) (3)
                                                              • Technical overview (Cont) (4)
                                                              • Build XFAs from Regex
                                                              • Build XFAs from Regex (Cont)
                                                              • Build XFAs from Regex (Cont) (2)
                                                              • Build XFAs from Regex (Cont) (3)
                                                              • Build XFAs from Regex (Cont) (4)
                                                              • Build XFAs from Regex (Cont) (5)
                                                              • Build XFAs from Regex (Cont) (6)
                                                              • Build XFAs from Regex (Cont) (7)
                                                              • Build XFAs from Regex (Cont) (8)
                                                              • Build XFAs from Regex (Cont) (9)
                                                              • Build XFAs from Regex (Cont) (10)
                                                              • Build XFAs from Regex (Cont) (11)
                                                              • Build XFAs from Regex (Cont) (12)
                                                              • Build XFAs from Regex (Cont) (13)
                                                              • Build XFAs from Regex (Cont) (14)
                                                              • Build XFAs from Regex (Cont) (15)
                                                              • Build XFAs from Regex (Cont) (16)
                                                              • Build XFAs from Regex (Cont) (17)
                                                              • Build XFAs from Regex (Cont) (18)
                                                              • Build XFAs from Regex (Cont) (19)
                                                              • Build XFAs from Regex (Cont) (20)
                                                              • Build XFAs from Regex (Cont) (21)
                                                              • Build XFAs from Regex (Cont) (22)
                                                              • Build XFAs from Regex (Cont) (23)
                                                              • Build XFAs from Regex (Cont) (24)
                                                              • Build XFAs from Regex (Cont) (25)
                                                              • Build XFAs from Regex (Cont) (26)
                                                              • Build XFAs from Regex (Cont) (27)
                                                              • Build XFAs from Regex (Cont) (28)
                                                              • Experimental Results
                                                              • Experimental Results (Cont)
                                                              • Experimental Results (Cont) (2)
                                                              • Conclusion

                                                                Build XFAs from Regex (Cont)

                                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                δlsquo G G G H H H H G H G

                                                                ( ) (G 3 5)

                                                                Frsquo

                                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                G G

                                                                3 5 7 3 5 6

                                                                32

                                                                Build XFAs from Regex (Cont)

                                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                δlsquo G G G H H H H G H G

                                                                ( ) (G 3 5)

                                                                Frsquo

                                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                G H

                                                                3 5 7 3 4 5

                                                                33

                                                                Build XFAs from Regex (Cont)

                                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                δlsquo G G G H H H H G H G

                                                                ( ) (G 3 5)

                                                                Frsquo

                                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                G G

                                                                3 5 6 3 5

                                                                34

                                                                Build XFAs from Regex (Cont)

                                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                δlsquo G G G H H H H G H G

                                                                ( ) (G 3 5)

                                                                Frsquo

                                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                G H

                                                                3 5 6 3 4 5

                                                                35

                                                                Build XFAs from Regex (Cont)

                                                                bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                δlsquo G G G H H H H G H G

                                                                ( ) (G 3 5)

                                                                Frsquo (G 3 5 6)

                                                                QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                G H

                                                                3 5 6 3 4 5

                                                                36

                                                                Build XFAs from Regex (Cont)

                                                                bull Compile to XFAbull Finding efficient implementations

                                                                37

                                                                Experimental Results

                                                                bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                                38

                                                                Experimental Results (Cont)

                                                                39

                                                                Experimental Results (Cont)

                                                                40

                                                                Conclusion

                                                                bull DFAs for regular expressions often blow up when combined

                                                                bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                bull Result compared to other feasible approaches reduce both time and space

                                                                41

                                                                • XFA Faster Signature Matching With Extended Automata
                                                                • Outline
                                                                • Introduction
                                                                • Technical overview
                                                                • Technical overview (Cont)
                                                                • Technical overview (Cont) (2)
                                                                • Technical overview (Cont) (3)
                                                                • Technical overview (Cont) (4)
                                                                • Build XFAs from Regex
                                                                • Build XFAs from Regex (Cont)
                                                                • Build XFAs from Regex (Cont) (2)
                                                                • Build XFAs from Regex (Cont) (3)
                                                                • Build XFAs from Regex (Cont) (4)
                                                                • Build XFAs from Regex (Cont) (5)
                                                                • Build XFAs from Regex (Cont) (6)
                                                                • Build XFAs from Regex (Cont) (7)
                                                                • Build XFAs from Regex (Cont) (8)
                                                                • Build XFAs from Regex (Cont) (9)
                                                                • Build XFAs from Regex (Cont) (10)
                                                                • Build XFAs from Regex (Cont) (11)
                                                                • Build XFAs from Regex (Cont) (12)
                                                                • Build XFAs from Regex (Cont) (13)
                                                                • Build XFAs from Regex (Cont) (14)
                                                                • Build XFAs from Regex (Cont) (15)
                                                                • Build XFAs from Regex (Cont) (16)
                                                                • Build XFAs from Regex (Cont) (17)
                                                                • Build XFAs from Regex (Cont) (18)
                                                                • Build XFAs from Regex (Cont) (19)
                                                                • Build XFAs from Regex (Cont) (20)
                                                                • Build XFAs from Regex (Cont) (21)
                                                                • Build XFAs from Regex (Cont) (22)
                                                                • Build XFAs from Regex (Cont) (23)
                                                                • Build XFAs from Regex (Cont) (24)
                                                                • Build XFAs from Regex (Cont) (25)
                                                                • Build XFAs from Regex (Cont) (26)
                                                                • Build XFAs from Regex (Cont) (27)
                                                                • Build XFAs from Regex (Cont) (28)
                                                                • Experimental Results
                                                                • Experimental Results (Cont)
                                                                • Experimental Results (Cont) (2)
                                                                • Conclusion

                                                                  Build XFAs from Regex (Cont)

                                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                  δlsquo G G G H H H H G H G

                                                                  ( ) (G 3 5)

                                                                  Frsquo

                                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                  G H

                                                                  3 5 7 3 4 5

                                                                  33

                                                                  Build XFAs from Regex (Cont)

                                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                  δlsquo G G G H H H H G H G

                                                                  ( ) (G 3 5)

                                                                  Frsquo

                                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                  G G

                                                                  3 5 6 3 5

                                                                  34

                                                                  Build XFAs from Regex (Cont)

                                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                  δlsquo G G G H H H H G H G

                                                                  ( ) (G 3 5)

                                                                  Frsquo

                                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                  G H

                                                                  3 5 6 3 4 5

                                                                  35

                                                                  Build XFAs from Regex (Cont)

                                                                  bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                  Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                  Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                  δlsquo G G G H H H H G H G

                                                                  ( ) (G 3 5)

                                                                  Frsquo (G 3 5 6)

                                                                  QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                  G H

                                                                  3 5 6 3 4 5

                                                                  36

                                                                  Build XFAs from Regex (Cont)

                                                                  bull Compile to XFAbull Finding efficient implementations

                                                                  37

                                                                  Experimental Results

                                                                  bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                                  38

                                                                  Experimental Results (Cont)

                                                                  39

                                                                  Experimental Results (Cont)

                                                                  40

                                                                  Conclusion

                                                                  bull DFAs for regular expressions often blow up when combined

                                                                  bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                  bull Result compared to other feasible approaches reduce both time and space

                                                                  41

                                                                  • XFA Faster Signature Matching With Extended Automata
                                                                  • Outline
                                                                  • Introduction
                                                                  • Technical overview
                                                                  • Technical overview (Cont)
                                                                  • Technical overview (Cont) (2)
                                                                  • Technical overview (Cont) (3)
                                                                  • Technical overview (Cont) (4)
                                                                  • Build XFAs from Regex
                                                                  • Build XFAs from Regex (Cont)
                                                                  • Build XFAs from Regex (Cont) (2)
                                                                  • Build XFAs from Regex (Cont) (3)
                                                                  • Build XFAs from Regex (Cont) (4)
                                                                  • Build XFAs from Regex (Cont) (5)
                                                                  • Build XFAs from Regex (Cont) (6)
                                                                  • Build XFAs from Regex (Cont) (7)
                                                                  • Build XFAs from Regex (Cont) (8)
                                                                  • Build XFAs from Regex (Cont) (9)
                                                                  • Build XFAs from Regex (Cont) (10)
                                                                  • Build XFAs from Regex (Cont) (11)
                                                                  • Build XFAs from Regex (Cont) (12)
                                                                  • Build XFAs from Regex (Cont) (13)
                                                                  • Build XFAs from Regex (Cont) (14)
                                                                  • Build XFAs from Regex (Cont) (15)
                                                                  • Build XFAs from Regex (Cont) (16)
                                                                  • Build XFAs from Regex (Cont) (17)
                                                                  • Build XFAs from Regex (Cont) (18)
                                                                  • Build XFAs from Regex (Cont) (19)
                                                                  • Build XFAs from Regex (Cont) (20)
                                                                  • Build XFAs from Regex (Cont) (21)
                                                                  • Build XFAs from Regex (Cont) (22)
                                                                  • Build XFAs from Regex (Cont) (23)
                                                                  • Build XFAs from Regex (Cont) (24)
                                                                  • Build XFAs from Regex (Cont) (25)
                                                                  • Build XFAs from Regex (Cont) (26)
                                                                  • Build XFAs from Regex (Cont) (27)
                                                                  • Build XFAs from Regex (Cont) (28)
                                                                  • Experimental Results
                                                                  • Experimental Results (Cont)
                                                                  • Experimental Results (Cont) (2)
                                                                  • Conclusion

                                                                    Build XFAs from Regex (Cont)

                                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                    δlsquo G G G H H H H G H G

                                                                    ( ) (G 3 5)

                                                                    Frsquo

                                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                    G G

                                                                    3 5 6 3 5

                                                                    34

                                                                    Build XFAs from Regex (Cont)

                                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                    δlsquo G G G H H H H G H G

                                                                    ( ) (G 3 5)

                                                                    Frsquo

                                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                    G H

                                                                    3 5 6 3 4 5

                                                                    35

                                                                    Build XFAs from Regex (Cont)

                                                                    bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                    Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                    Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                    δlsquo G G G H H H H G H G

                                                                    ( ) (G 3 5)

                                                                    Frsquo (G 3 5 6)

                                                                    QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                    G H

                                                                    3 5 6 3 4 5

                                                                    36

                                                                    Build XFAs from Regex (Cont)

                                                                    bull Compile to XFAbull Finding efficient implementations

                                                                    37

                                                                    Experimental Results

                                                                    bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                                    38

                                                                    Experimental Results (Cont)

                                                                    39

                                                                    Experimental Results (Cont)

                                                                    40

                                                                    Conclusion

                                                                    bull DFAs for regular expressions often blow up when combined

                                                                    bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                    bull Result compared to other feasible approaches reduce both time and space

                                                                    41

                                                                    • XFA Faster Signature Matching With Extended Automata
                                                                    • Outline
                                                                    • Introduction
                                                                    • Technical overview
                                                                    • Technical overview (Cont)
                                                                    • Technical overview (Cont) (2)
                                                                    • Technical overview (Cont) (3)
                                                                    • Technical overview (Cont) (4)
                                                                    • Build XFAs from Regex
                                                                    • Build XFAs from Regex (Cont)
                                                                    • Build XFAs from Regex (Cont) (2)
                                                                    • Build XFAs from Regex (Cont) (3)
                                                                    • Build XFAs from Regex (Cont) (4)
                                                                    • Build XFAs from Regex (Cont) (5)
                                                                    • Build XFAs from Regex (Cont) (6)
                                                                    • Build XFAs from Regex (Cont) (7)
                                                                    • Build XFAs from Regex (Cont) (8)
                                                                    • Build XFAs from Regex (Cont) (9)
                                                                    • Build XFAs from Regex (Cont) (10)
                                                                    • Build XFAs from Regex (Cont) (11)
                                                                    • Build XFAs from Regex (Cont) (12)
                                                                    • Build XFAs from Regex (Cont) (13)
                                                                    • Build XFAs from Regex (Cont) (14)
                                                                    • Build XFAs from Regex (Cont) (15)
                                                                    • Build XFAs from Regex (Cont) (16)
                                                                    • Build XFAs from Regex (Cont) (17)
                                                                    • Build XFAs from Regex (Cont) (18)
                                                                    • Build XFAs from Regex (Cont) (19)
                                                                    • Build XFAs from Regex (Cont) (20)
                                                                    • Build XFAs from Regex (Cont) (21)
                                                                    • Build XFAs from Regex (Cont) (22)
                                                                    • Build XFAs from Regex (Cont) (23)
                                                                    • Build XFAs from Regex (Cont) (24)
                                                                    • Build XFAs from Regex (Cont) (25)
                                                                    • Build XFAs from Regex (Cont) (26)
                                                                    • Build XFAs from Regex (Cont) (27)
                                                                    • Build XFAs from Regex (Cont) (28)
                                                                    • Experimental Results
                                                                    • Experimental Results (Cont)
                                                                    • Experimental Results (Cont) (2)
                                                                    • Conclusion

                                                                      Build XFAs from Regex (Cont)

                                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                      δlsquo G G G H H H H G H G

                                                                      ( ) (G 3 5)

                                                                      Frsquo

                                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                      G H

                                                                      3 5 6 3 4 5

                                                                      35

                                                                      Build XFAs from Regex (Cont)

                                                                      bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                      Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                      Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                      δlsquo G G G H H H H G H G

                                                                      ( ) (G 3 5)

                                                                      Frsquo (G 3 5 6)

                                                                      QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                      G H

                                                                      3 5 6 3 4 5

                                                                      36

                                                                      Build XFAs from Regex (Cont)

                                                                      bull Compile to XFAbull Finding efficient implementations

                                                                      37

                                                                      Experimental Results

                                                                      bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                                      38

                                                                      Experimental Results (Cont)

                                                                      39

                                                                      Experimental Results (Cont)

                                                                      40

                                                                      Conclusion

                                                                      bull DFAs for regular expressions often blow up when combined

                                                                      bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                      bull Result compared to other feasible approaches reduce both time and space

                                                                      41

                                                                      • XFA Faster Signature Matching With Extended Automata
                                                                      • Outline
                                                                      • Introduction
                                                                      • Technical overview
                                                                      • Technical overview (Cont)
                                                                      • Technical overview (Cont) (2)
                                                                      • Technical overview (Cont) (3)
                                                                      • Technical overview (Cont) (4)
                                                                      • Build XFAs from Regex
                                                                      • Build XFAs from Regex (Cont)
                                                                      • Build XFAs from Regex (Cont) (2)
                                                                      • Build XFAs from Regex (Cont) (3)
                                                                      • Build XFAs from Regex (Cont) (4)
                                                                      • Build XFAs from Regex (Cont) (5)
                                                                      • Build XFAs from Regex (Cont) (6)
                                                                      • Build XFAs from Regex (Cont) (7)
                                                                      • Build XFAs from Regex (Cont) (8)
                                                                      • Build XFAs from Regex (Cont) (9)
                                                                      • Build XFAs from Regex (Cont) (10)
                                                                      • Build XFAs from Regex (Cont) (11)
                                                                      • Build XFAs from Regex (Cont) (12)
                                                                      • Build XFAs from Regex (Cont) (13)
                                                                      • Build XFAs from Regex (Cont) (14)
                                                                      • Build XFAs from Regex (Cont) (15)
                                                                      • Build XFAs from Regex (Cont) (16)
                                                                      • Build XFAs from Regex (Cont) (17)
                                                                      • Build XFAs from Regex (Cont) (18)
                                                                      • Build XFAs from Regex (Cont) (19)
                                                                      • Build XFAs from Regex (Cont) (20)
                                                                      • Build XFAs from Regex (Cont) (21)
                                                                      • Build XFAs from Regex (Cont) (22)
                                                                      • Build XFAs from Regex (Cont) (23)
                                                                      • Build XFAs from Regex (Cont) (24)
                                                                      • Build XFAs from Regex (Cont) (25)
                                                                      • Build XFAs from Regex (Cont) (26)
                                                                      • Build XFAs from Regex (Cont) (27)
                                                                      • Build XFAs from Regex (Cont) (28)
                                                                      • Experimental Results
                                                                      • Experimental Results (Cont)
                                                                      • Experimental Results (Cont) (2)
                                                                      • Conclusion

                                                                        Build XFAs from Regex (Cont)

                                                                        bull Compile to XFAbull Alg 3 data determinization for NXFAs

                                                                        Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

                                                                        Drsquo 3 5 3 4 5 3 5 7 3 5 6

                                                                        δlsquo G G G H H H H G H G

                                                                        ( ) (G 3 5)

                                                                        Frsquo (G 3 5 6)

                                                                        QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

                                                                        G H

                                                                        3 5 6 3 4 5

                                                                        36

                                                                        Build XFAs from Regex (Cont)

                                                                        bull Compile to XFAbull Finding efficient implementations

                                                                        37

                                                                        Experimental Results

                                                                        bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                                        38

                                                                        Experimental Results (Cont)

                                                                        39

                                                                        Experimental Results (Cont)

                                                                        40

                                                                        Conclusion

                                                                        bull DFAs for regular expressions often blow up when combined

                                                                        bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                        bull Result compared to other feasible approaches reduce both time and space

                                                                        41

                                                                        • XFA Faster Signature Matching With Extended Automata
                                                                        • Outline
                                                                        • Introduction
                                                                        • Technical overview
                                                                        • Technical overview (Cont)
                                                                        • Technical overview (Cont) (2)
                                                                        • Technical overview (Cont) (3)
                                                                        • Technical overview (Cont) (4)
                                                                        • Build XFAs from Regex
                                                                        • Build XFAs from Regex (Cont)
                                                                        • Build XFAs from Regex (Cont) (2)
                                                                        • Build XFAs from Regex (Cont) (3)
                                                                        • Build XFAs from Regex (Cont) (4)
                                                                        • Build XFAs from Regex (Cont) (5)
                                                                        • Build XFAs from Regex (Cont) (6)
                                                                        • Build XFAs from Regex (Cont) (7)
                                                                        • Build XFAs from Regex (Cont) (8)
                                                                        • Build XFAs from Regex (Cont) (9)
                                                                        • Build XFAs from Regex (Cont) (10)
                                                                        • Build XFAs from Regex (Cont) (11)
                                                                        • Build XFAs from Regex (Cont) (12)
                                                                        • Build XFAs from Regex (Cont) (13)
                                                                        • Build XFAs from Regex (Cont) (14)
                                                                        • Build XFAs from Regex (Cont) (15)
                                                                        • Build XFAs from Regex (Cont) (16)
                                                                        • Build XFAs from Regex (Cont) (17)
                                                                        • Build XFAs from Regex (Cont) (18)
                                                                        • Build XFAs from Regex (Cont) (19)
                                                                        • Build XFAs from Regex (Cont) (20)
                                                                        • Build XFAs from Regex (Cont) (21)
                                                                        • Build XFAs from Regex (Cont) (22)
                                                                        • Build XFAs from Regex (Cont) (23)
                                                                        • Build XFAs from Regex (Cont) (24)
                                                                        • Build XFAs from Regex (Cont) (25)
                                                                        • Build XFAs from Regex (Cont) (26)
                                                                        • Build XFAs from Regex (Cont) (27)
                                                                        • Build XFAs from Regex (Cont) (28)
                                                                        • Experimental Results
                                                                        • Experimental Results (Cont)
                                                                        • Experimental Results (Cont) (2)
                                                                        • Conclusion

                                                                          Build XFAs from Regex (Cont)

                                                                          bull Compile to XFAbull Finding efficient implementations

                                                                          37

                                                                          Experimental Results

                                                                          bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                                          38

                                                                          Experimental Results (Cont)

                                                                          39

                                                                          Experimental Results (Cont)

                                                                          40

                                                                          Conclusion

                                                                          bull DFAs for regular expressions often blow up when combined

                                                                          bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                          bull Result compared to other feasible approaches reduce both time and space

                                                                          41

                                                                          • XFA Faster Signature Matching With Extended Automata
                                                                          • Outline
                                                                          • Introduction
                                                                          • Technical overview
                                                                          • Technical overview (Cont)
                                                                          • Technical overview (Cont) (2)
                                                                          • Technical overview (Cont) (3)
                                                                          • Technical overview (Cont) (4)
                                                                          • Build XFAs from Regex
                                                                          • Build XFAs from Regex (Cont)
                                                                          • Build XFAs from Regex (Cont) (2)
                                                                          • Build XFAs from Regex (Cont) (3)
                                                                          • Build XFAs from Regex (Cont) (4)
                                                                          • Build XFAs from Regex (Cont) (5)
                                                                          • Build XFAs from Regex (Cont) (6)
                                                                          • Build XFAs from Regex (Cont) (7)
                                                                          • Build XFAs from Regex (Cont) (8)
                                                                          • Build XFAs from Regex (Cont) (9)
                                                                          • Build XFAs from Regex (Cont) (10)
                                                                          • Build XFAs from Regex (Cont) (11)
                                                                          • Build XFAs from Regex (Cont) (12)
                                                                          • Build XFAs from Regex (Cont) (13)
                                                                          • Build XFAs from Regex (Cont) (14)
                                                                          • Build XFAs from Regex (Cont) (15)
                                                                          • Build XFAs from Regex (Cont) (16)
                                                                          • Build XFAs from Regex (Cont) (17)
                                                                          • Build XFAs from Regex (Cont) (18)
                                                                          • Build XFAs from Regex (Cont) (19)
                                                                          • Build XFAs from Regex (Cont) (20)
                                                                          • Build XFAs from Regex (Cont) (21)
                                                                          • Build XFAs from Regex (Cont) (22)
                                                                          • Build XFAs from Regex (Cont) (23)
                                                                          • Build XFAs from Regex (Cont) (24)
                                                                          • Build XFAs from Regex (Cont) (25)
                                                                          • Build XFAs from Regex (Cont) (26)
                                                                          • Build XFAs from Regex (Cont) (27)
                                                                          • Build XFAs from Regex (Cont) (28)
                                                                          • Experimental Results
                                                                          • Experimental Results (Cont)
                                                                          • Experimental Results (Cont) (2)
                                                                          • Conclusion

                                                                            Experimental Results

                                                                            bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

                                                                            38

                                                                            Experimental Results (Cont)

                                                                            39

                                                                            Experimental Results (Cont)

                                                                            40

                                                                            Conclusion

                                                                            bull DFAs for regular expressions often blow up when combined

                                                                            bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                            bull Result compared to other feasible approaches reduce both time and space

                                                                            41

                                                                            • XFA Faster Signature Matching With Extended Automata
                                                                            • Outline
                                                                            • Introduction
                                                                            • Technical overview
                                                                            • Technical overview (Cont)
                                                                            • Technical overview (Cont) (2)
                                                                            • Technical overview (Cont) (3)
                                                                            • Technical overview (Cont) (4)
                                                                            • Build XFAs from Regex
                                                                            • Build XFAs from Regex (Cont)
                                                                            • Build XFAs from Regex (Cont) (2)
                                                                            • Build XFAs from Regex (Cont) (3)
                                                                            • Build XFAs from Regex (Cont) (4)
                                                                            • Build XFAs from Regex (Cont) (5)
                                                                            • Build XFAs from Regex (Cont) (6)
                                                                            • Build XFAs from Regex (Cont) (7)
                                                                            • Build XFAs from Regex (Cont) (8)
                                                                            • Build XFAs from Regex (Cont) (9)
                                                                            • Build XFAs from Regex (Cont) (10)
                                                                            • Build XFAs from Regex (Cont) (11)
                                                                            • Build XFAs from Regex (Cont) (12)
                                                                            • Build XFAs from Regex (Cont) (13)
                                                                            • Build XFAs from Regex (Cont) (14)
                                                                            • Build XFAs from Regex (Cont) (15)
                                                                            • Build XFAs from Regex (Cont) (16)
                                                                            • Build XFAs from Regex (Cont) (17)
                                                                            • Build XFAs from Regex (Cont) (18)
                                                                            • Build XFAs from Regex (Cont) (19)
                                                                            • Build XFAs from Regex (Cont) (20)
                                                                            • Build XFAs from Regex (Cont) (21)
                                                                            • Build XFAs from Regex (Cont) (22)
                                                                            • Build XFAs from Regex (Cont) (23)
                                                                            • Build XFAs from Regex (Cont) (24)
                                                                            • Build XFAs from Regex (Cont) (25)
                                                                            • Build XFAs from Regex (Cont) (26)
                                                                            • Build XFAs from Regex (Cont) (27)
                                                                            • Build XFAs from Regex (Cont) (28)
                                                                            • Experimental Results
                                                                            • Experimental Results (Cont)
                                                                            • Experimental Results (Cont) (2)
                                                                            • Conclusion

                                                                              Experimental Results (Cont)

                                                                              39

                                                                              Experimental Results (Cont)

                                                                              40

                                                                              Conclusion

                                                                              bull DFAs for regular expressions often blow up when combined

                                                                              bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                              bull Result compared to other feasible approaches reduce both time and space

                                                                              41

                                                                              • XFA Faster Signature Matching With Extended Automata
                                                                              • Outline
                                                                              • Introduction
                                                                              • Technical overview
                                                                              • Technical overview (Cont)
                                                                              • Technical overview (Cont) (2)
                                                                              • Technical overview (Cont) (3)
                                                                              • Technical overview (Cont) (4)
                                                                              • Build XFAs from Regex
                                                                              • Build XFAs from Regex (Cont)
                                                                              • Build XFAs from Regex (Cont) (2)
                                                                              • Build XFAs from Regex (Cont) (3)
                                                                              • Build XFAs from Regex (Cont) (4)
                                                                              • Build XFAs from Regex (Cont) (5)
                                                                              • Build XFAs from Regex (Cont) (6)
                                                                              • Build XFAs from Regex (Cont) (7)
                                                                              • Build XFAs from Regex (Cont) (8)
                                                                              • Build XFAs from Regex (Cont) (9)
                                                                              • Build XFAs from Regex (Cont) (10)
                                                                              • Build XFAs from Regex (Cont) (11)
                                                                              • Build XFAs from Regex (Cont) (12)
                                                                              • Build XFAs from Regex (Cont) (13)
                                                                              • Build XFAs from Regex (Cont) (14)
                                                                              • Build XFAs from Regex (Cont) (15)
                                                                              • Build XFAs from Regex (Cont) (16)
                                                                              • Build XFAs from Regex (Cont) (17)
                                                                              • Build XFAs from Regex (Cont) (18)
                                                                              • Build XFAs from Regex (Cont) (19)
                                                                              • Build XFAs from Regex (Cont) (20)
                                                                              • Build XFAs from Regex (Cont) (21)
                                                                              • Build XFAs from Regex (Cont) (22)
                                                                              • Build XFAs from Regex (Cont) (23)
                                                                              • Build XFAs from Regex (Cont) (24)
                                                                              • Build XFAs from Regex (Cont) (25)
                                                                              • Build XFAs from Regex (Cont) (26)
                                                                              • Build XFAs from Regex (Cont) (27)
                                                                              • Build XFAs from Regex (Cont) (28)
                                                                              • Experimental Results
                                                                              • Experimental Results (Cont)
                                                                              • Experimental Results (Cont) (2)
                                                                              • Conclusion

                                                                                Experimental Results (Cont)

                                                                                40

                                                                                Conclusion

                                                                                bull DFAs for regular expressions often blow up when combined

                                                                                bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                                bull Result compared to other feasible approaches reduce both time and space

                                                                                41

                                                                                • XFA Faster Signature Matching With Extended Automata
                                                                                • Outline
                                                                                • Introduction
                                                                                • Technical overview
                                                                                • Technical overview (Cont)
                                                                                • Technical overview (Cont) (2)
                                                                                • Technical overview (Cont) (3)
                                                                                • Technical overview (Cont) (4)
                                                                                • Build XFAs from Regex
                                                                                • Build XFAs from Regex (Cont)
                                                                                • Build XFAs from Regex (Cont) (2)
                                                                                • Build XFAs from Regex (Cont) (3)
                                                                                • Build XFAs from Regex (Cont) (4)
                                                                                • Build XFAs from Regex (Cont) (5)
                                                                                • Build XFAs from Regex (Cont) (6)
                                                                                • Build XFAs from Regex (Cont) (7)
                                                                                • Build XFAs from Regex (Cont) (8)
                                                                                • Build XFAs from Regex (Cont) (9)
                                                                                • Build XFAs from Regex (Cont) (10)
                                                                                • Build XFAs from Regex (Cont) (11)
                                                                                • Build XFAs from Regex (Cont) (12)
                                                                                • Build XFAs from Regex (Cont) (13)
                                                                                • Build XFAs from Regex (Cont) (14)
                                                                                • Build XFAs from Regex (Cont) (15)
                                                                                • Build XFAs from Regex (Cont) (16)
                                                                                • Build XFAs from Regex (Cont) (17)
                                                                                • Build XFAs from Regex (Cont) (18)
                                                                                • Build XFAs from Regex (Cont) (19)
                                                                                • Build XFAs from Regex (Cont) (20)
                                                                                • Build XFAs from Regex (Cont) (21)
                                                                                • Build XFAs from Regex (Cont) (22)
                                                                                • Build XFAs from Regex (Cont) (23)
                                                                                • Build XFAs from Regex (Cont) (24)
                                                                                • Build XFAs from Regex (Cont) (25)
                                                                                • Build XFAs from Regex (Cont) (26)
                                                                                • Build XFAs from Regex (Cont) (27)
                                                                                • Build XFAs from Regex (Cont) (28)
                                                                                • Experimental Results
                                                                                • Experimental Results (Cont)
                                                                                • Experimental Results (Cont) (2)
                                                                                • Conclusion

                                                                                  Conclusion

                                                                                  bull DFAs for regular expressions often blow up when combined

                                                                                  bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

                                                                                  bull Result compared to other feasible approaches reduce both time and space

                                                                                  41

                                                                                  • XFA Faster Signature Matching With Extended Automata
                                                                                  • Outline
                                                                                  • Introduction
                                                                                  • Technical overview
                                                                                  • Technical overview (Cont)
                                                                                  • Technical overview (Cont) (2)
                                                                                  • Technical overview (Cont) (3)
                                                                                  • Technical overview (Cont) (4)
                                                                                  • Build XFAs from Regex
                                                                                  • Build XFAs from Regex (Cont)
                                                                                  • Build XFAs from Regex (Cont) (2)
                                                                                  • Build XFAs from Regex (Cont) (3)
                                                                                  • Build XFAs from Regex (Cont) (4)
                                                                                  • Build XFAs from Regex (Cont) (5)
                                                                                  • Build XFAs from Regex (Cont) (6)
                                                                                  • Build XFAs from Regex (Cont) (7)
                                                                                  • Build XFAs from Regex (Cont) (8)
                                                                                  • Build XFAs from Regex (Cont) (9)
                                                                                  • Build XFAs from Regex (Cont) (10)
                                                                                  • Build XFAs from Regex (Cont) (11)
                                                                                  • Build XFAs from Regex (Cont) (12)
                                                                                  • Build XFAs from Regex (Cont) (13)
                                                                                  • Build XFAs from Regex (Cont) (14)
                                                                                  • Build XFAs from Regex (Cont) (15)
                                                                                  • Build XFAs from Regex (Cont) (16)
                                                                                  • Build XFAs from Regex (Cont) (17)
                                                                                  • Build XFAs from Regex (Cont) (18)
                                                                                  • Build XFAs from Regex (Cont) (19)
                                                                                  • Build XFAs from Regex (Cont) (20)
                                                                                  • Build XFAs from Regex (Cont) (21)
                                                                                  • Build XFAs from Regex (Cont) (22)
                                                                                  • Build XFAs from Regex (Cont) (23)
                                                                                  • Build XFAs from Regex (Cont) (24)
                                                                                  • Build XFAs from Regex (Cont) (25)
                                                                                  • Build XFAs from Regex (Cont) (26)
                                                                                  • Build XFAs from Regex (Cont) (27)
                                                                                  • Build XFAs from Regex (Cont) (28)
                                                                                  • Experimental Results
                                                                                  • Experimental Results (Cont)
                                                                                  • Experimental Results (Cont) (2)
                                                                                  • Conclusion

                                                                                    top related

                                                                                    Hypersensitivity reactions dr somesh - MICROBIOLOGY
                                                                                    Education
                                                                                    BETA 90 SFA/XFA - Kemppi - scis.com.au
                                                                                    Documents
                                                                                    X-FEM for Abaqus (XFA) Toolkit for Automated Crack … for.....
                                                                                    Documents
                                                                                    XFA xeometría da forma arquitectónica
                                                                                    Documents
                                                                                    Who's afraid of XFA?
                                                                                    Technology
                                                                                    diesel locomotive works training report by somesh dwivedi
                                                                                    Engineering
                                                                                    Service Parts+Ml 3050,+Ml 3051nd Xfa
                                                                                    Documents
                                                                                    Delta Delta+ 90 XFA
                                                                                    Documents
                                                                                    Tips for style sheets and Flash objects in XFA forms -...
                                                                                    Documents
                                                                                    XFA: Faster Signature Matching With Extended...
                                                                                    Documents
                                                                                    p179_182 Somesh Bhadra Mathabhanga Cooch Behar...
                                                                                    Documents
                                                                                    PrInt COPy xFA revolutionizing today’s Workspace ·...
                                                                                    Documents
                                                                                    Behavior-Based Malware Detection Somesh Jha University of...
                                                                                    Documents
                                                                                    Estan Terias
                                                                                    Documents
                                                                                    somesh pg seminar2001
                                                                                    Documents
                                                                                    Ppt Ips- Denny Xfa- 06
                                                                                    Documents