Top Banner
XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy 2008 Presenter: Yu-Hao, Tseng Date: 2014/01/15 1
41

XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Dec 18, 2015

Download

Documents

Samantha Lloyd
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

XFA Faster Signature Matching With Extended AutomataAuthor Randy Smith Cristian Estan and Somesh Jha

Publisher IEEE Symposium on Security and Privacy 2008

Presenter Yu-Hao Tseng

Date 20140115

1

Outline

bull Introductionbull Technical overviewbull Build XFAs from Regexbull Experimental Results

2

Introduction

bull In this paper their primary goal is to improve the time and space efficiency of signature matching in network intrusion detection systems (NIDS)

bull To achieve their goal they introduce extended finite automata (XFAs) which augment traditional FSAs with a finite scratch memory used to remember various types of information relevant to the progress of signature matching

3

Technical overview

bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

4

Technical overview (Cont)

bull where all and are distinct strings

5

Technical overview (Cont)

bull where all and are distinct stringsbull use a single bit of scratch memory

6

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 2: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Outline

bull Introductionbull Technical overviewbull Build XFAs from Regexbull Experimental Results

2

Introduction

bull In this paper their primary goal is to improve the time and space efficiency of signature matching in network intrusion detection systems (NIDS)

bull To achieve their goal they introduce extended finite automata (XFAs) which augment traditional FSAs with a finite scratch memory used to remember various types of information relevant to the progress of signature matching

3

Technical overview

bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

4

Technical overview (Cont)

bull where all and are distinct strings

5

Technical overview (Cont)

bull where all and are distinct stringsbull use a single bit of scratch memory

6

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 3: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Introduction

bull In this paper their primary goal is to improve the time and space efficiency of signature matching in network intrusion detection systems (NIDS)

bull To achieve their goal they introduce extended finite automata (XFAs) which augment traditional FSAs with a finite scratch memory used to remember various types of information relevant to the progress of signature matching

3

Technical overview

bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

4

Technical overview (Cont)

bull where all and are distinct strings

5

Technical overview (Cont)

bull where all and are distinct stringsbull use a single bit of scratch memory

6

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 4: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Technical overview

bull For NIDS signatures REs overlap or subsume each otherbull Matching progress interleavedbull Many distinct combination of reachable states

bull Two signaturesbull where all and are distinct stringsbull which consists of all strings of length n

4

Technical overview (Cont)

bull where all and are distinct strings

5

Technical overview (Cont)

bull where all and are distinct stringsbull use a single bit of scratch memory

6

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 5: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Technical overview (Cont)

bull where all and are distinct strings

5

Technical overview (Cont)

bull where all and are distinct stringsbull use a single bit of scratch memory

6

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 6: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Technical overview (Cont)

bull where all and are distinct stringsbull use a single bit of scratch memory

6

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 7: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Technical overview (Cont)bull which consists of all strings of length n

bull use a counter

7

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 8: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Technical overview (Cont)bull XFA = DFAs+ auxiliary variables1048708

bull Changes shape of automatabull Tames state space explosion

8

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 9: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex

bull Annotating regular expressionsbull Compiling to an XFAbull From parse trees to NXFAsbull From NXFAs to XFAs

bull ε ndasheliminationbull determinizing transitionsbull data determinization

bull Finding efficient implementations

9

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 10: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Annotate Signaturebull New operators change parse tree and add domain values 1048708

bull Parallel concatenation ( ) adds a bit1048708bull Breaks up RE into string-like componentsbull Set a bit when the left operand acceptsbull Test the bit when the right operand accepts

bull ex abcd =gt abcd

10

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 11: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull XFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ QtimesΣrarrQ is the transition functionbull D is the finite set of values in the data domainbull QtimesΣtimesDrarrD is the per transition update function which defines how the

data value is updated on every transitionbull (q0d0) is the initial configuration which consists of an initial state q0 and

an initial data value d0bull F QtimesD is the set of accepting configurationssube

11

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 12: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Definitions

bull NXFA is a 7-tuple (Q D Σ δ ( )F)bull Q is the set of statesbull Σ is the set of inputs (input alphabet)bull δ Qtimes(Σ ε )timesQ is the nondeterministic relation describing the allowed sube cup

transitionsbull D is the finite set of values in the data domainbull δ rarr is the nondeterministic update function (or update relation) which

defines how the data value is updated on every transitionbull QD0 QtimesD is the set of initial configurations of the NXFAsubebull F QtimesD is the set of accepting configurationssube

12

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 13: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

13

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 14: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

14

ａ

ｂ

sum

1

[ a]

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 15: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull From parse trees to NXFAs

bull Ex ab[^a]1 =gt ab[^a]1

15

1

2

0

3

4

bit = 0cnt = 0 sum

a

bsum

[^a]

ε

ε

ε

cnt++

if (bit == 1 ampamp cnt = 1) accept()

bit = 1

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 16: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

16

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 17: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

17

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 18: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 1 ε -elimination for NXFAs

18

120576

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 19: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo

A C

Frsquo

19

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 20: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C

A C

Frsquo

A C

A C

20

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 21: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 1) (C 0) (C 2)

δlsquo A C A C A C A B C

A C

Frsquo

A B C

A C

21

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 22: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0)Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C

A C

Frsquo

A B C

A B C

22

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 23: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C

A C

Frsquo

A C

A B C

23

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 24: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo

A C

A B C

24

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 25: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 2 determinizing transitions for NXFAs

Qrsquo A C A B C (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 0) (C 1) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (B 1) (A 0) (C 0) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (B 1) (C 2) (C 2) (C 1) (A 0) (A 0) (A 0) (C 0) (C 0) (C 0) (C 2) (C 1)

Drsquo (A 0) (B 0) (C 0) (C 2)

δlsquo A C A C A C A B C A B C A B C A B C A C A B C A C

A C

Frsquo (A C (C 1)) (A B C (C 1))

A C

A B C

25

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 26: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

26

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 27: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5

Drsquo 3 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5)

G G

35 35

27

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 28: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

G H

35 3 4 5

28

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 29: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5Drsquo 3 5 3 4 5

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5)

H H

3 4 5 3 4 5

29

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 30: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5 7

30

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 31: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7)

H G

3 4 5 3 5

31

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 32: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 7 3 5 6

32

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 33: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 7 3 4 5

33

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 34: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G G

3 5 6 3 5

34

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 35: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

35

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 36: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Alg 3 data determinization for NXFAs

Q G H 3 5 3 5 3 5 7 3 5 6 3 5 6 3 5 3 5 3 4 5 3 5 7 3 4 5 3 5 6 3 4 5 3 4 5 3 4 5 3 4 5 3 5 7 3 4 5 3 5

Drsquo 3 5 3 4 5 3 5 7 3 5 6

δlsquo G G G H H H H G H G

( ) (G 3 5)

Frsquo (G 3 5 6)

QD (G 3 5) (H 3 4 5) (G 3 5 7) (G 3 5 6)

G H

3 5 6 3 4 5

36

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 37: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Build XFAs from Regex (Cont)

bull Compile to XFAbull Finding efficient implementations

37

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 38: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Experimental Results

bull 1450 Regular expressions extracted from Snort HTTPbull Characteristics of combined XFA1048708bull 41994 total states =gt 42 MBbull 195 bits (~25 bytes) of aux memorybull Instruction memory 35 MB

38

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 39: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Experimental Results (Cont)

39

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 40: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Experimental Results (Cont)

40

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion
Page 41: XFA : Faster Signature Matching With Extended Automata Author: Randy Smith, Cristian Estan and Somesh Jha Publisher: IEEE Symposium on Security and Privacy.

Conclusion

bull DFAs for regular expressions often blow up when combined

bull XFA = DFAs+ auxiliary variables1048708bull Changes shape of automatabull Tames state space explosion

bull Result compared to other feasible approaches reduce both time and space

41

  • XFA Faster Signature Matching With Extended Automata
  • Outline
  • Introduction
  • Technical overview
  • Technical overview (Cont)
  • Technical overview (Cont) (2)
  • Technical overview (Cont) (3)
  • Technical overview (Cont) (4)
  • Build XFAs from Regex
  • Build XFAs from Regex (Cont)
  • Build XFAs from Regex (Cont) (2)
  • Build XFAs from Regex (Cont) (3)
  • Build XFAs from Regex (Cont) (4)
  • Build XFAs from Regex (Cont) (5)
  • Build XFAs from Regex (Cont) (6)
  • Build XFAs from Regex (Cont) (7)
  • Build XFAs from Regex (Cont) (8)
  • Build XFAs from Regex (Cont) (9)
  • Build XFAs from Regex (Cont) (10)
  • Build XFAs from Regex (Cont) (11)
  • Build XFAs from Regex (Cont) (12)
  • Build XFAs from Regex (Cont) (13)
  • Build XFAs from Regex (Cont) (14)
  • Build XFAs from Regex (Cont) (15)
  • Build XFAs from Regex (Cont) (16)
  • Build XFAs from Regex (Cont) (17)
  • Build XFAs from Regex (Cont) (18)
  • Build XFAs from Regex (Cont) (19)
  • Build XFAs from Regex (Cont) (20)
  • Build XFAs from Regex (Cont) (21)
  • Build XFAs from Regex (Cont) (22)
  • Build XFAs from Regex (Cont) (23)
  • Build XFAs from Regex (Cont) (24)
  • Build XFAs from Regex (Cont) (25)
  • Build XFAs from Regex (Cont) (26)
  • Build XFAs from Regex (Cont) (27)
  • Build XFAs from Regex (Cont) (28)
  • Experimental Results
  • Experimental Results (Cont)
  • Experimental Results (Cont) (2)
  • Conclusion

Related Documents
BETA 90 SFA/XFA - kemppi.nt-rt.ru · 3,2А·ч, зарядное устройство для аккумулятора, расходомер. Beta 90 XFA Комплект респиратора

BETA 90 SFA/XFA - kemppi.nt-rt.ru · 3,2А·ч,...

Category: Documents
XFA : Faster Signature Matching With Extended Automata

XFA : Faster Signature Matching With Extended Automata

Category: Documents
HRM- SYBBA Somesh-Swap

HRM- SYBBA Somesh-Swap

Category: Documents
XFA xeometría da forma arquitectónica

XFA xeometría da forma arquitectónica

Category: Documents
FORIMMEDIATE$RELEASE$ Meritech… XFA Footwear Hygiene... · FORIMMEDIATE$RELEASE$ Meritech$Introduces$its$Newest$Boot$&$Footwear$Hygiene$System$—$The$XFA$ Fully$Automated,HighVolumeWalkHThroughSanitizingProductforFoodProduction$

FORIMMEDIATE$RELEASE$ Meritech… XFA Footwear...

Category: Documents
somesh jain markeing

somesh jain markeing

Category: Documents
Delta Delta+ 90 XFA

Delta Delta+ 90 XFA

Category: Documents
Estan Terias

Estan Terias

Category: Documents
BETA 90 SFA/XFA - Kemppi - scis.com.au

BETA 90 SFA/XFA - Kemppi - scis.com.au

Category: Documents
tesi somesh 2 - | UniSelinus · Mr. Somesh R. Doddi (candidate#UNISE0707IT) A DISSERTATION Presented to the Department of Biology and Life Sciences program at Selinus University Faculty

tesi somesh 2 - | UniSelinus · Mr. Somesh R. Doddi...

Category: Documents
Estan Dares

Estan Dares

Category: Documents
Beta e90 PFA Beta e90 SFA Beta e90 XFA e90 ... - s.siteapi.org

Beta e90 PFA Beta e90 SFA Beta e90 XFA e90 ... -...

Category: Documents