Emerging patterns based classifier

Con

tras

t Dat

a M

inin

g: M

etho

ds

and

App

licat

ions

Jam

es B

aile

y, N

ICT

A V

icto

ria L

abor

ator

y an

d T

he U

nive

rsity

of M

elbo

urne

Guo

zhu

Don

g, W

right

Sta

te U

nive

rsity

Pre

sent

ed a

t the

IEE

E In

tern

atio

nal C

onfe

renc

e on

Dat

a M

inin

g (I

CD

M),

Oct

ober

28-

31 2

007

An

up to

dat

e ve

rsio

n of

this

tuto

rial i

s av

aila

ble

at h

ttp://

ww

w.c

sse.

unim

elb.

edu.

au/~

jbai

ley/

cont

rast

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g2

Con

tras

t dat

a m

inin

g -

Wha

t is

it ?

Co

ntr

ast

-``

To

com

pare

or

appr

aise

in

resp

ect t

o di

ffere

nces

’’ (M

erria

m W

ebst

er D

ictio

nary

)

Co

ntr

ast

dat

a m

inin

g-

The

min

ing

of

patte

rns

and

mod

els

cont

rast

ing

two

or

mor

e cl

asse

s/co

nditi

ons.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g3

Con

tras

t Dat

a M

inin

g -

Why

?

``S

omet

imes

it’s

goo

d to

con

trast

wha

t you

lik

e w

ith s

omet

hing

els

e. I

t mak

es y

ou

appr

ecia

te it

eve

n m

ore’

’D

arby

Con

ley,

Get

Fuz

zy, 2

001

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g4

Wha

t can

be

cont

rast

ed ?

Obj

ects

at d

iffer

ent t

ime

perio

ds

``C

ompa

re IC

DM

pap

ers

publ

ishe

d in

200

6-20

07

vers

us th

ose

in 2

004-

2005

’’

Obj

ects

for

diffe

rent

spa

tiall

ocat

ions

``F

ind

the

dist

ingu

ishi

ng fe

atur

es o

f loc

atio

n x

for

hum

an D

NA

, ver

sus

loca

tion

xfo

r m

ouse

DN

A’’

Obj

ects

acr

oss

diffe

rent

cla

sses

``F

ind

the

diffe

renc

es b

etw

een

peop

le w

ith

brow

n ha

ir, v

ersu

s th

ose

with

blo

nde

hair’

’

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g5

Wha

t can

be

cont

rast

ed ?

Con

t.

Obj

ects

with

ina

clas

s``

With

in th

e ac

adem

ic p

rofe

ssio

n, th

ere

are

few

pe

ople

old

er th

an 8

0’’ (

rarit

y)``

With

in th

e ac

adem

ic p

rofe

ssio

n, th

ere

are

no r

ich

peop

le’’

(hol

es)

``W

ithin

com

pute

r sc

ienc

e, m

ost o

f the

pap

ers

com

e fr

om U

SA

or

Eur

ope’

’ (ab

unda

nce)

Obj

ect p

ositi

ons

in a

ran

king

``F

ind

the

diffe

renc

es b

etw

een

high

and

low

inco

me

earn

ers’

’

Com

bina

tions

of t

he a

bove

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g6

Alte

rnat

ive

nam

es fo

r co

ntra

st d

ata

min

ing

Con

tras

t={c

hang

e, d

iffer

ence

, dis

crim

inat

or,

clas

sific

atio

n ru

le, …

}

Con

tras

t dat

a m

inin

g is

rel

ated

to to

pics

suc

h as

:C

hang

e de

tect

ion,

cla

ss b

ased

ass

ocia

tion

rule

s, c

ontr

ast s

ets,

co

ncep

t drif

t, di

ffere

nce

dete

ctio

n, d

iscr

imin

ativ

e pa

ttern

s,

(dis

)sim

ilarit

y in

dex,

em

ergi

ng p

atte

rns,

gra

dien

t min

ing,

hig

h co

nfid

ence

pat

tern

s, (

in)f

requ

ent p

atte

rns,

top

k pa

ttern

s,…

…

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g7

Cha

ract

eris

tics

of c

ontr

ast d

ata

min

ing

App

lied

to m

ultiv

aria

te d

ata

Obj

ects

may

be

rela

tiona

l, se

quen

tial,

grap

hs, m

odel

s, c

lass

ifier

s, c

ombi

natio

ns

of th

ese

Use

rs m

ay w

ant e

ither

To

find

mul

tiple

cont

rast

s (a

ll, o

r to

p k)

A s

ingl

em

easu

re fo

r co

mpa

rison

•``

The

deg

ree

of d

iffer

ence

bet

wee

n th

e gr

oups

(or

m

odel

s) is

0.7

’’

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g8

Con

tras

t cha

ract

eris

tics

Con

t.

Rep

rese

ntat

ion

of c

ontr

asts

is im

port

ant.

N

eeds

to b

eIn

terp

reta

ble,

non

red

unda

nt, p

oten

tially

act

iona

ble,

ex

pres

sive

Tra

ctab

leto

com

pute

Qua

lity

of c

ontr

asts

is a

lso

impo

rtan

t. N

eed

Sta

tistic

al s

igni

fican

ce, w

hich

can

be

mea

sure

d in

m

ultip

le w

ays

Abi

lity

to r

ank

cont

rast

s is

des

irabl

e, e

spec

ially

for

clas

sific

atio

n

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g9

How

is c

ontr

ast d

ata

min

ing

used

?

Dom

ain

unde

rsta

ndin

g``

You

ng c

hild

ren

with

dia

bete

s ha

ve a

gre

ater

ris

k of

hos

pita

l ad

mis

sion

, com

pare

d to

the

rest

of t

he p

opul

atio

n

Use

d fo

r bu

ildin

g cl

assi

fiers

Man

y di

ffere

nt te

chni

ques

-to

be

cove

red

late

rA

lso

used

for

wei

ghtin

gan

d ra

nkin

gin

stan

ces

Use

d in

con

stru

ctio

n of

syn

thet

icin

stan

ces

Goo

d fo

r ra

recl

asse

s

Use

d fo

r al

ertin

g, n

otifi

catio

n an

d m

onito

ring

``T

ell m

e w

hen

the

diss

imila

rity

inde

x fa

lls b

elow

0.3

’’

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g10

Goa

ls o

f thi

s tu

toria

l

Pro

vide

an

over

view

of c

ontr

ast d

ata

min

ing

Brin

g to

geth

er r

esul

ts fr

om a

num

ber

of

disp

arat

e ar

eas.

Min

ing

for

diffe

rent

type

s of

dat

a•

Rel

atio

nal,

sequ

ence

, gra

ph, m

odel

s, …

Cla

ssifi

catio

nus

ing

disc

rimin

atin

g pa

ttern

s

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g11

By

the

end

of th

is tu

toria

l you

will

be

abl

e to

…

Und

erst

and

som

e pr

inci

pal t

echn

ique

s fo

r re

pres

entin

gco

ntra

sts

and

eval

uatin

gth

eir

qual

ityA

ppre

ciat

e so

me

min

ing

tech

niqu

es fo

r co

ntra

st d

isco

very

U

nder

stan

d te

chni

ques

for

usin

g co

ntra

sts

in c

lass

ifica

tion

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g12

Don

’t ha

ve ti

me

to c

over

..

Str

ing

algo

rithm

sC

onne

ctio

ns to

wor

k in

indu

ctiv

e lo

gic

prog

ram

min

gT

ree-

base

d co

ntra

sts

Cha

nges

in d

ata

stre

ams

Fre

quen

t pat

tern

alg

orith

ms

Con

nect

ions

to g

ranu

lar

com

putin

g…

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g13

Out

line

of th

e tu

toria

l

Bas

ic n

otio

ns a

nd u

niva

riate

cont

rast

sP

atte

rn a

nd r

ule

base

d co

ntra

sts

Con

tras

t pat

tern

bas

ed c

lass

ifica

tion

Con

tras

ts fo

r ra

re c

lass

dat

aset

sD

ata

cube

con

tras

tsS

eque

nce

base

d co

ntra

sts

Gra

ph b

ased

con

tras

tsM

odel

bas

ed c

ontr

asts

Com

mon

them

es +

ope

n pr

oble

ms

+ s

umm

ary

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g14

Bas

ic n

otio

ns a

nd u

niva

riate

case

Fea

ture

sel

ectio

nan

d fe

atur

e si

gnifi

canc

ete

sts

can

be th

ough

t of a

s a

basi

c co

ntra

st d

ata

min

ing

activ

ity.

``T

ell m

e th

e di

scrim

inat

ing

feat

ures

’’ •

Wou

ld li

ke a

sin

gle

qual

itym

easu

re•

Use

ful f

or fe

atur

e ra

nkin

g

Em

phas

is is

less

on

findi

ngth

e co

ntra

st a

nd

mor

e on

eva

luat

ing

its p

ower

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g15

Sam

ple

Fea

ture

-Cla

ss D

atas

et

Hap

py ☺

150

9004

3325

4327

9006

1005

ID

…..

……

Hap

py ☺

120

Hap

py ☺

137

Sad

200

Cla

ssH

eigh

t (cm

)

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g16

Dis

crim

inat

ive

pow

er

Can

ass

ess

disc

rimin

ativ

e po

wer

of H

eigh

tfe

atur

e by

Info

rmat

ion

mea

sure

s(s

igna

l to

nois

e, in

form

atio

n ga

in r

atio

, …)

Sta

tistic

al te

sts

(t-t

est,

Kol

mog

orov

-Sm

irnov

, Chi

sq

uare

d, W

ilcox

onra

nk s

um, …

). A

sses

sing

w

heth

er

•T

he m

ean

of e

ach

clas

s is

the

sam

e•

The

sam

ples

for

each

cla

ss c

ome

from

the

sam

e di

strib

utio

n•

How

wel

l a d

atas

et fi

ts a

hyp

othe

sis

No

sing

le te

st is

bes

t in

all s

ituat

ions

!

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g17

Exa

mpl

e D

iscr

imin

ativ

e P

ower

T

est -

Wilc

oxon

Ran

k S

um

Sup

pose

n1

happ

y, a

nd n

2sa

d in

stan

ces

Sor

t the

inst

ance

s ac

cord

ing

to h

eigh

t val

ue:

h 1<

= h

2<

= h

3<=

… h

n 1+

n 2A

ssig

n a

rank

to e

ach

inst

ance

, ind

icat

ing

how

man

y in

stan

ces

in th

e ot

her

clas

s ar

e le

ss.

For

x in

cla

ss A

For

eac

h cl

ass

Com

pute

the

Ran

ksum

=S

um(r

anks

of a

ll its

inst

ance

s)N

ull H

ypot

hesi

s: T

he in

stan

ces

are

from

the

sam

e di

strib

utio

nC

onsu

lt st

atis

tical

sig

nific

ance

tabl

e to

det

erm

ine

whe

ther

val

ue

of R

anks

umis

sig

nific

ant

Ran

k(x)

=|{

y: c

lass

(y)<

>A

and

hei

ght(

y)<

heig

ht(x

)}|

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g18

Ran

k S

um C

alcu

latio

n E

xam

ple

0H

appy

☺12

081

6

1S

ad

15

041

5

1H

appy

☺17

732

1

2S

ad

19

066

0

2S

ad

21

048

13

Hap

py ☺

220

324

Ran

kC

lass

Hei

ght(

cm)

ID Hap

py:R

ankS

um=

3+1+

0=4

Sad

:Ran

kSum

=2+

2+1=

5

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g19

Wilc

oxon

Ran

k S

um T

estC

ont.

Non

par

amet

ric (

no n

orm

al d

istr

ibut

ion

assu

mpt

ion)

Req

uire

s an

ord

erin

g on

the

attr

ibut

e va

lues

Sca

led

valu

e of

Ran

ksum

is e

quiv

alen

t to

area

unde

r R

OC

curv

e fo

r us

ing

the

sele

cted

feat

ure

as a

cla

ssifi

erTrue Positive Rate 0 %

100%

Fals

e Po

sitiv

e R

ate

0 %10

0%

Ran

ksum

(n1*

n 2)

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g20

Dis

crim

inat

ing

with

attr

ibut

e va

lues

Can

alte

rnat

ivel

y fo

cus

on s

igni

fican

ce o

f at

trib

ute

valu

es, w

ith e

ither

1) F

requ

ency

/infr

eque

ncy

(hig

h/lo

w c

ount

s)F

requ

ent i

n on

e cl

ass

and

infr

eque

nt in

the

othe

r.

•T

here

are

50

happ

y pe

ople

of h

eigh

t 200

cm a

nd o

nly

2 sa

d pe

ople

of h

eigh

t 200

cm

2) R

atio

(hig

h ra

tio o

f sup

port

)A

ppea

rs X

tim

es m

ore

in o

ne c

lass

than

the

othe

r•

The

re a

re 2

5 tim

es m

ore

happ

y pe

ople

of h

eigh

t 200

cm

than

sad

peo

ple

of h

eigh

t 200

cm

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g21

Attr

ibut

e/F

eatu

re C

onve

rsio

n

Pos

sibl

e to

form

a n

ew b

inar

y fe

atur

e ba

sed

on a

ttrib

ute

valu

e an

d th

en a

pply

fe

atur

e si

gnifi

canc

e te

sts

Blu

r di

stin

ctio

n be

twee

n at

trib

ute

and

attr

ibut

e va

lue

Hap

py ☺

…N

oY

es

Cla

ss…

200c

m15

0cm

Sad

…Y

esN

o

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g22

Dis

crim

inat

ing

Attr

ibut

e V

alue

s in

a

Dat

a S

trea

m

Det

ectin

g ch

ange

s in

attr

ibut

e va

lues

is a

n im

port

ant f

ocus

in d

ata

stre

ams

Ofte

n fo

cus

on u

niva

riate

cont

rast

s fo

r ef

ficie

ncy

reas

ons

Fin

ding

whe

nch

ange

occ

urs

(non

sta

tiona

ry

stre

am).

F

indi

ng th

e m

agni

tude

of th

e ch

ange

. E.g

. How

big

is

the

dist

ance

bet

wee

n tw

o sa

mpl

es o

f the

str

eam

?U

sefu

l for

sig

nalin

g ne

cess

ityfo

r m

odel

upd

ate

or a

n im

pend

ing

faul

t or

criti

cal e

vent

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g23

Odd

s ra

tio a

nd R

isk

ratio

Can

be

used

for

com

parin

g or

mea

surin

g ef

fect

siz

eU

sefu

l for

bin

ary

data

Wel

l kno

wn

in c

linic

al c

onte

xts

Can

als

o be

use

d fo

r qu

ality

eva

luat

ion

of

mul

tivar

iate

con

tras

ts (

will

see

late

r)A

sim

ple

exam

ple

give

n ne

xt

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g24

Odd

s an

d ris

k ra

tio C

ont.

4321ID

……

No

Mal

e

No

Fem

ale

Yes

Mal

e

Exp

osed

(e

vent

)G

ende

r (f

eatu

re)

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g25

Odd

s R

atio

Exa

mpl

e

Sup

pose

we

have

100

men

and

100

wom

en,

and

70 m

en a

nd 1

0 w

omen

hav

e be

en e

xpos

edO

dds

of e

xpos

ure(

mal

e)=

0.7/

0.3=

2.33

Odd

s of

exp

osur

e(fe

mal

e)=

0.1/

0.9=

0.11

Odd

s ra

tio=

2.33

/.11=

21.2

Mal

es h

ave

21.2

tim

es th

e od

ds o

f exp

osur

e th

an fe

mal

esIn

dica

tes

expo

sure

is m

uch

mor

e lik

ely

for

mal

es th

an fo

r fe

mal

es

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g26

Rel

ativ

e R

isk

Exa

mpl

e

Sup

pose

we

have

100

men

and

100

wom

en,

and

70 m

en a

nd 1

0 w

omen

hav

e be

en e

xpos

edR

elat

ive

risk

of e

xpos

ure

(mal

e)=

70/1

00=

0.7

Rel

ativ

e ris

k of

exp

osur

e(fe

mal

e)=

10/1

00=

0.1

The

rel

ativ

e ris

k=0.

7/0.

1=7

Men

7 ti

mes

mor

e lik

ely

to b

e ex

pose

d th

an

wom

en

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g27

Pat

tern

/Rul

e B

ased

Con

tras

ts

Ove

rvie

w o

f ``r

elat

iona

l’’ c

ontr

ast p

atte

rn m

inin

g E

mer

ging

pat

tern

s an

d m

inin

gJu

mpi

ng e

mer

ging

pat

tern

s C

ompu

tatio

nal c

ompl

exity

B

orde

r di

ffere

ntia

l alg

orith

m•

Gen

e cl

ub +

bor

der

diffe

rent

ial

•In

crem

enta

l min

ing

Tre

e ba

sed

algo

rithm

Pro

ject

ion

base

d al

gorit

hmZ

BD

D b

ased

alg

orith

m

Bio

info

rmat

icap

plic

atio

n: c

ance

r st

udy

on m

icro

arra

yge

ne e

xpre

ssio

n da

ta

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g28

Ove

rvie

w

Cla

ss b

ased

ass

ocia

tion

rule

s (C

aiet

al 9

0, L

iu e

t al 9

8, ..

.)

Ver

sion

spa

ces

(Mitc

hell

77)

Em

ergi

ng p

atte

rns

(Don

g+Li

99)

–m

any

algo

rithm

s (la

ter)

Con

tras

t set

min

ing

(Bay

+P

azza

ni99

, Web

b et

al 0

3)

Odd

s ra

tio r

ules

& d

elta

dis

crim

inat

ive

EP

(Li e

t al 0

5, L

i et

al 0

7)

MD

L ba

sed

cont

rast

(Sie

bes,

KD

D07

)

Usi

ng s

tatis

tical

mea

sure

s to

eva

luat

e gr

oup

diffe

renc

es

(Hild

erm

an+

Pec

kman

05, W

ebb

07)

Spa

tial c

ontr

ast p

atte

rns

(Aru

nasa

lam

et a

l 05)

……

see

ref

eren

ces

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g29

Cla

ssifi

catio

n/A

ssoc

iatio

n R

ules

Cla

ssifi

catio

n ru

les

--sp

ecia

l ass

ocia

tion

rule

s (w

ith ju

st o

ne it

em –

clas

s --

on R

HS

):X

C

(s,

c)

•X

is a

pat

tern

, •

C is

a c

lass

,

•s

is s

uppo

rt,

•c

is c

onfid

ence

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g30

Ver

sion

Spa

ce (

Mitc

hell)

Ver

sion

spa

ce: t

he s

et o

f all

patte

rns

cons

iste

nt w

ith

give

n (D

+,D

-) –

patte

rns

sepa

ratin

g D

+, D

-.T

he s

pace

is d

elim

ited

by a

spe

cific

& a

gen

eral

boun

dary

. U

sefu

l for

sea

rchi

ng th

e tr

ue h

ypot

hesi

s, w

hich

lies

som

ewhe

re

b/w

the

two

boun

darie

s.A

ddin

g +

veex

ampl

es to

D+

mak

es th

e sp

ecifi

c bo

unda

ry m

ore

gene

ral;

addi

ng -

veex

ampl

es to

D-

mak

es th

e ge

nera

l bo

unda

ry m

ore

spec

ific.

Com

mon

pat

tern

/hyp

othe

sis

lang

uage

ope

rato

rs:

conj

unct

ion,

dis

junc

tion

Pat

tern

s/hy

poth

eses

are

cris

p; n

eed

to b

e ge

nera

lized

to

dea

l with

per

cent

ages

; har

d to

dea

l with

noi

se in

dat

a

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g31

ST

UC

CO

, MA

GN

UM

OP

US

for

cont

rast

pa

ttern

min

ing

ST

UC

CO

(B

ay+

Paz

zani

99)

Min

ing

cont

rast

pat

tern

s X

(ca

lled

cont

rast

set

s) b

etw

een

k>=

2 gr

oups

: |su

ppi(X

) –

supp

j(X)|

>=

min

Diff

Use

Chi

2 to

mea

sure

sta

tistic

al s

igni

fican

ce o

f con

tras

t pat

tern

s•

sign

ifica

nce

cut-

off t

hres

hold

s ch

ange

, bas

ed o

n th

e le

vel o

f the

no

de a

nd th

e lo

cal n

umbe

r of

con

tras

t pat

tern

s M

ax-M

iner

like

sea

rch

stra

tegy

, plu

s so

me

prun

ing

tech

niqu

es

MA

GN

UM

OP

US

(W

ebb

01)

An

asso

ciat

ion

rule

min

ing

met

hod,

usi

ng M

ax-M

iner

like

ap

proa

ch (

prop

osed

bef

ore,

and

inde

pend

ently

of,

Max

-Min

er)

Can

min

e co

ntra

st p

atte

rns

(by

limiti

ng R

HS

to a

cla

ss)

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g32

Con

tras

t pat

tern

s vs

deci

sion

tree

ba

sed

rule

s

It ha

s be

en r

ecog

nize

d by

sev

eral

aut

hors

(e.

g.

Bay

+P

azza

ni99

) th

at

rule

s ge

nera

tion

from

dec

isio

n tr

ees

can

be g

ood

cont

rast

pat

tern

s,

but m

ay m

iss

man

y go

od c

ontr

ast p

atte

rns.

Diff

eren

t con

tras

t set

min

ing

algo

rithm

s ha

ve

diffe

rent

thre

shol

dsS

ome

have

min

sup

port

thre

shol

dS

ome

have

no

min

sup

port

thre

shol

d; lo

w s

uppo

rt

patte

rns

may

be

usef

ul fo

r cl

assi

ficat

ion

etc

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g33

Em

ergi

ng P

atte

rns

Em

ergi

ng P

atte

rns

(EP

s) a

re c

ontr

ast p

atte

rns

betw

een

two

clas

ses

of d

ata

who

se s

uppo

rt c

hang

es s

igni

fican

tly b

etw

een

the

two

clas

ses.

Cha

nge

sign

ifica

nce

can

be d

efin

ed b

y:

If su

pp2(

X)/

supp

1(X

) =

infin

ity, t

hen

X is

a ju

mpi

ng E

P.

jum

ping

EP

occ

urs

in s

ome

mem

bers

of o

ne c

lass

but

nev

er

occu

rs in

the

othe

r cl

ass.

Con

junc

tive

lang

uage

; ext

ensi

on to

dis

junc

tive

EP

late

r

sim

ilar

to R

iskR

atio

; +:

allo

win

g pa

ttern

s w

ith

smal

l ove

rall

supp

ort

big

supp

ort r

atio

:su

pp2(

X)/

supp

1(X

) >

= m

inR

atio

big

supp

ort d

iffer

ence

:|s

upp2

(X)

–su

pp1(

X)|

>=

min

Diff

(as

defin

ed b

y B

ay+P

azza

ni99

)

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g34

A ty

pica

l EP

in th

e M

ushr

oom

dat

aset

The

Mus

hroo

m d

atas

et c

onta

ins

two

clas

ses:

edi

ble

and

pois

onou

s.E

ach

data

tupl

eha

s se

vera

l fea

ture

s su

ch a

s: o

dor,

rin

g-nu

mbe

r, s

talk

-sur

face

-bel

low

-rin

g, e

tc.

Con

side

r th

e pa

ttern

{o

dor

= n

one,

st

alk-

surf

ace-

belo

w-r

ing

= s

moo

th,

ring-

num

ber

= o

ne}

Its s

uppo

rt in

crea

ses

from

0.2

% in

the

pois

onou

s cl

ass

to

57.6

% in

the

edib

le c

lass

(a

grow

th r

ate

of 2

88).

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g35

Exa

mpl

e E

P in

mic

roar

ray

data

for

canc

er

Nor

mal

Tis

sues

Can

cer

Tis

sues

Jum

ping

EP

: Pat

tern

s w

/ hig

h su

ppor

t rat

io b

/w d

ata

clas

ses

E.G

. {g1

=L,

g2=

H,g

3=L}

; sup

pN=

50%

, sup

pC=

0

LH

HL

HL

LH

LL

HL

HL

HL

g4g3

g2g1

LH

HH

HL

LL

HH

HL

HL

HH

g4g3

g2g1

binn

ed

data

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g36

Top

sup

port

min

imal

jum

ping

EP

s fo

r co

lon

canc

er

Col

on C

ance

r E

Ps

{1+

4-

112+

113

+}

100%

{1+

4-

113+

116

+}

100%

{1+

4-

113+

221

+}

100%

{1+

4-

113+

696

+}

100%

{1+

108

-11

2+ 1

13+

} 10

0%{1

+ 1

08-

113+

116

+}

100%

{4-

108-

112+

113

+}

100%

{4-

109+

113

+ 7

00+

} 10

0%{4

-11

0+ 1

12+

113

+}

100%

{4-

112+

113

+ 7

00+

} 10

0%{4

-11

3+ 1

17+

700

+}

100%

{1+

6+

8-

700+

} 97

.5%

Col

on N

orm

al E

Ps

{12-

21-

35+

40+

137

+ 2

54+

} 10

0%{1

2-35

+ 4

0+ 7

1-13

7+ 2

54+

} 10

0%{2

0-21

-35

+ 1

37+

254

+}

100%

{20-

35+

71-

137+

254

+}

100%

{5-

35+

137

+ 1

77+

} 95

.5%

{5-

35+

137

+ 2

54+

} 95

.5%

{5-

35+

137

+ 4

19-}

95.

5%{5

-13

7+ 1

77+

309

+}

95.5

%{5

-13

7+ 2

54+

309

+}

95.5

%{7

-21

-33

+ 3

5+ 6

9+}

95.5

%{7

-21

-33

+ 6

9+ 3

09+

} 95

.5%

{7-

21-

33+

69+

126

1+}

95.5

%

EP

s fr

om

Mao

+D

ong

2005

(g

ene

club

+

bord

er-d

iff).

Col

on c

ance

r da

tase

t (A

lon

et a

l, 19

99 (

PN

AS

)): 4

0 ca

ncer

tiss

ues,

22

nor

mal

tiss

ues.

200

0 ge

nes

The

se E

Ps

have

95%

--1

00%

sup

port

in o

ne

clas

s bu

t 0%

sup

port

in

the

othe

r cl

ass.

Min

imal

: Eac

h pr

oper

su

bset

occ

urs

in b

oth

clas

ses.

Ver

y fe

w 1

00%

sup

port

EP

s.

The

re a

re ~

1000

item

s w

ith s

upp

>=

80%

.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g37

A p

oten

tial u

se o

f min

imal

jum

ping

EP

sM

inim

al ju

mpi

ng E

Ps

for

norm

altis

sues

Pro

perly

exp

ress

ed g

ene

grou

ps im

port

ant f

or n

orm

al c

ell f

unct

ioni

ng, b

ut

dest

roye

d in

all

colo

n ca

ncer

tiss

ues

Res

tore

thes

e ?c

ure

colo

n ca

ncer

?

Min

imal

jum

ping

EP

s fo

r ca

ncer

tissu

es

Bad

gen

e gr

oups

that

occ

ur in

som

e ca

ncer

tiss

ues

but n

ever

occ

ur in

nor

mal

tissu

es

Dis

rupt

thes

e ?c

ure

colo

n ca

ncer

?

? P

ossi

ble

targ

ets

for

drug

des

ign

?Li

+W

ong

2002

pro

pose

d “g

ene

ther

apy

usin

g E

P”

idea

: the

rapy

aim

s to

des

troy

ba

d JE

P &

res

tore

goo

d JE

P

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g38

Use

fuln

ess

of E

mer

ging

Pat

tern

sE

Ps

are

usef

ul

for

build

ing

high

ly a

ccur

ate

and

robu

st c

lass

ifier

s, a

nd fo

r im

prov

ing

othe

r ty

pes

of c

lass

ifier

s fo

r di

scov

erin

g po

wer

ful d

istin

guis

hing

feat

ures

bet

wee

n da

tase

ts.

Like

oth

er p

atte

rns

com

pose

d of

con

junc

tive

com

bina

tion

of e

lem

ents

, EP

s ar

e ea

sy fo

r pe

ople

to u

nder

stan

d an

d us

e di

rect

ly.

EP

s ca

n al

so c

aptu

re p

atte

rns

abou

t cha

nge

over

tim

e.

Pap

ers

usin

g E

P te

chni

ques

in C

ance

r C

ell (

cove

r, 3

/02)

.E

mer

ging

Pat

tern

s ha

ve b

een

appl

ied

in m

edic

al a

pplic

atio

ns fo

r

diag

nosi

ng a

cute

Lym

phob

last

icLe

ukem

ia.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g39

The

land

scap

e of

EP

s on

the

supp

ort p

lane

, an

d ch

alle

nges

for

min

ing

O1

1

Sup

D2

(X)

Sup D1 (X)

C BA

•E

P m

inR

atio

cons

trai

nt is

ne

ither

mon

oton

ic n

or a

nti-

mon

oton

ic (

but e

xcep

tions

ex

ist f

or s

peci

al c

ases

)•

Req

uire

s sm

alle

r su

ppor

t th

resh

olds

than

thos

e us

ed

for

freq

uent

pat

tern

min

ing

Land

scap

e of

EP

sC

halle

nges

for

EP

m

inin

g

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g40

Odd

s R

atio

and

Rel

ativ

e R

isk

Pat

tern

s [L

i and

Won

g P

OD

S06

]

May

use

odd

s ra

tio/r

elat

ive

risk

to

eval

uate

com

poun

d fa

ctor

s as

wel

lM

ay b

e no

sin

gle

fact

or w

ith h

igh

rela

tive

risk

or o

dds

ratio

, but

a c

ombi

natio

n of

fact

ors

•R

elat

ive

risk

patte

rns

-S

imila

r to

em

ergi

ng

patte

rns

•R

isk

diffe

renc

e pa

ttern

s -

Sim

ilar

to c

ontr

ast s

ets

•O

dds

ratio

pat

tern

s

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g41

Min

ing

Pat

tern

s w

ith H

igh

Odd

s R

atio

or

Rel

ativ

e R

isk

Spa

ce o

f odd

s ra

tio p

atte

rns

and

rela

tive

risk

patte

rns

are

not c

onve

x in

gen

eral

Can

bec

ome

conv

ex, i

f str

atifi

ed in

to

plat

eaus

, bas

ed o

n su

ppor

t lev

els

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g42

EP

Min

ing

Alg

orith

ms

Com

plex

ity r

esul

t (W

ang

et a

l 05)

Bor

der-

diffe

rent

ial a

lgor

ithm

(D

ong+

Li 9

9)G

ene

club

+ b

orde

r di

ffere

ntia

l (M

ao+

Don

g 05

)C

onst

rain

t-ba

sed

appr

oach

(Z

hang

et a

l 00)

Tre

e-ba

sed

appr

oach

(B

aile

y et

al 0

2,

Fan

+K

otag

iri02

)P

roje

ctio

n ba

sed

algo

rithm

(B

aile

y el

al 0

3)Z

BD

D b

ased

met

hod

(Loe

kito

+B

aile

y06

).

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g43

Com

plex

ity r

esul

t

The

com

plex

ity o

f fin

ding

em

ergi

ng

patte

rns

(eve

n th

ose

with

the

high

est

freq

uenc

y) is

MA

X S

NP

-har

d.

Thi

s im

plie

s th

at p

olyn

omia

l tim

e ap

prox

imat

ion

sche

mes

do

not e

xist

for

the

prob

lem

unl

ess

P=

NP

.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g44

Bor

ders

are

con

cise

rep

rese

ntat

ions

of

conv

ex c

olle

ctio

ns o

f ite

mse

ts

< m

inB

={1

2,13

}, m

axB

={1

2345

,124

56}>

123,

123

412

124,

123

5

123

45

125,

124

5

124

56

126,

124

6

1313

4, 1

256

135,

134

5

A c

olle

ctio

n S

is c

onve

x:

If fo

r al

l X,Y

,Z (

X in

S, Y

in

S, X

sub

set Z

sub

set

Y)

Z in

S.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g45

Bor

der-

Diff

eren

tial A

lgor

ithm

<{{

}},{

1234

}> -

<{{}

},{2

3,24

,34}

>=

<{1

,234

},{1

234}

>{}{}

1,,

22, , 3,

43,

412

, 13,

14

,

, 2

3, 2

423

, 24

, , 3434

123,

124

, 134

,234

1234

Goo

d fo

r: J

umpi

ng E

Ps;

EP

s in

“re

ctan

gle

regi

ons,

” …

Alg

orith

m:

•U

se ite

rations

of

expan

sion &

m

inim

izat

ion o

f “p

roduct

s” o

f diffe

rence

s

•U

se t

ree

to s

pee

d

up m

inim

izat

ion

•F

ind

min

imal

sub

sets

of 1

234

that

are

not

sub

sets

of 2

3, 2

4, 3

4.

•{1

,234

} =

min

({1

,4}

X {

1,3}

X {

1,2}

)

Itera

tive

expa

nsio

n &

min

imiz

atio

n ca

n be

vi

ewed

as

optim

ized

Ber

ge h

yper

grap

htr

ansv

ersa

l alg

orith

m

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g46

Gen

e cl

ub +

Bor

der

Diff

eren

tial

Bor

der-

diffe

rent

ial c

an h

andl

e up

to 7

5 at

trib

utes

(us

ing

2003

PC

)F

or m

icro

arra

yge

ne e

xpre

ssio

n da

ta, t

here

are

th

ousa

nds

of g

enes

. (M

ao+

Don

g 05

) us

ed b

orde

r-di

ffere

ntia

l afte

r fin

ding

m

any

gene

clu

bs -

-on

e ge

ne c

lub

per

gene

.A

gen

e cl

ub is

a s

et o

f k g

enes

str

ongl

y co

rrel

ated

with

a

give

n ge

ne a

nd th

e cl

asse

s.

Som

e E

Ps

disc

over

ed u

sing

this

met

hod

wer

e sh

own

earli

er. D

isco

vere

d m

ore

EP

s w

ith n

ear

100%

sup

port

in

canc

er o

r no

rmal

, inv

olvi

ng m

any

diffe

rent

gen

es. M

uch

bette

r th

an e

arlie

r re

sults

.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g47

Tre

e-ba

sed

algo

rithm

for

JEP

min

ing

Use

tree

to c

ompr

ess

data

and

pat

tern

s.T

ree

is s

imila

r to

FP

tree

, but

it s

tore

s tw

o co

unts

per

no

de (

one

per

clas

s)an

d us

es d

iffer

ent i

tem

ord

erin

gN

odes

with

non

-zer

o su

ppor

t for

pos

itive

cla

ss a

nd z

ero

supp

ort f

or n

egat

ive

clas

s ar

e ca

lled

base

nod

es.

For

eve

ry b

ase

node

, the

pat

h’s

item

seti

s a

pote

ntia

l JE

P. G

athe

r ne

gativ

e da

ta c

onta

inin

g ro

ot it

em a

nd it

em

for

base

d no

des

on th

e pa

th. C

all b

orde

r di

ffere

ntia

l.Ite

m o

rder

ing

is im

port

ant.

Hyb

rid (

supp

ort r

atio

or

derin

g fir

st fo

r a

perc

enta

ge o

f ite

ms,

freq

uenc

y or

derin

g fo

r ot

her

item

s) is

bes

t.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g48

Pro

ject

ion

base

d al

gorit

hmF

orm

dat

aset

H c

onta

inin

g th

e di

ffere

nces

{p

-ni|

i=1…

k}.

p is

a p

ositi

ve tr

ansa

ctio

n, n

1, …

, nk

are

nega

tive

tran

sact

ions

.F

ind

min

imal

tran

sver

als

of h

yper

grap

hH

. i.e

. The

sm

alle

st s

ets

inte

rsec

ting

ever

y ed

ge (

equi

vale

nt to

th

e sm

alle

st s

ubse

ts o

f p n

ot c

onta

ined

in a

ny n

i).Le

t x1<

…<

xmbe

incr

easi

ng it

em fr

eque

ncy

(in

H)

orde

ring.

For

i=1

to m

le

t Hxi

be H

with

all

item

s y

> x

i pro

ject

ed o

ut &

al

l tra

nsac

tions

con

tain

ing

xi r

emov

ed (

data

pr

ojec

tion)

.re

mov

e no

n m

inim

al tr

ansa

ctio

ns in

Hxi

.if

Hxi

is s

mal

l, ap

ply

bord

er d

iffer

entia

l O

ther

wis

e, a

pply

the

algo

rithm

on

Hxi

.

Let H

be:

a b

c d

(edg

e 1)

b e

d

(edg

e 2)

b c

e

(edg

e 3)

c d

e

(edg

e 4)

Item

ord

erin

g:

a <

b <

c <

d <

e

Ha

is H

with

all

item

s >

a (

red

item

s)pr

ojec

ted

out

and

also

edg

e w

ith a

rem

oved

, so

Ha=

{}.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g49

ZB

DD

bas

ed a

lgor

ithm

to

min

e di

sjun

ctiv

e em

ergi

ng p

atte

rns

Dis

jun

ctiv

e E

mer

gin

g P

atte

rns:

allo

win

gdi

sjun

ctio

n as

wel

l as

conj

unct

ion

of

sim

ple

attr

ibut

e co

nditi

ons.

e.g

. Pre

cip

itat

ion

=(

gt-n

orm

OR

lt-no

rm)

AN

D

In

tern

al d

isco

lora

tio

n =

( br

own

OR

bla

ck )

Gen

eral

izat

ion

of E

Ps

ZB

DD

bas

ed a

lgor

ithm

use

s Z

ero

Sup

pres

sed

Bin

ary

Dec

isio

n D

iagr

am fo

r ef

ficie

ntly

min

ing

disj

unct

ive

EP

s.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g50

Pop

ular

in b

oole

anS

AT

sol

vers

and

rel

iabi

lity

eng.

Can

onic

al D

AG

rep

rese

ntat

ions

of b

oole

anfo

rmul

ae

No

de

shar

ing

: ide

ntic

al n

odes

are

sha

red

Cac

hin

g p

rin

cip

le: p

ast c

ompu

tatio

n re

sults

are

aut

omat

ical

ly s

tore

d an

d ca

n be

ret

rieve

dE

ffici

ent B

DD

impl

emen

tatio

ns a

vaila

ble,

e.g

. CU

DD

(U

of C

olor

ado)

Bin

ary

Dec

isio

n D

iagr

ams

(BD

Ds)

c ad

10

root

f = (c

Λa)

v (d

Λa)

c

ad

10

a1

0

0

10

dotte

d (o

r 0)

edg

e: d

on’t

link

the

node

s (in

form

ulae

)

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g51

ZB

DD

Rep

rese

ntat

ion

of It

emse

ts

Zer

o-s

up

pre

ssed

BD

D, Z

BD

D: A

BD

D v

aria

nt fo

r m

anip

ulat

ion

of it

em

com

bina

tions

E.g

. Bui

ldin

g a

ZB

DD

for

{{a,b,c,e}

,{a,b,d,e

},{b,c,d}

}

Ord

erin

g : c

<d

<a

< e

< b

c a e b

10

d a e b

10

c a e b

10

d

={{a,b,c,e}

}{{a,b,d,e}

}{{a,b,c,e},{a,b,d,e

}}U

z{{b,c,d}

}U

z=

{{a,b,c,e}

,{a,b,d,e

},{b,c,d

}} c dd a

e

b

10

c d b

10

Uz

Uz=

ZBD

D s

et-u

nion

Uz

==

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g52

ZB

DD

bas

ed m

inin

g ex

ampl

eU

se s

olid

pat

hs in

ZB

DD

(Dn)

to g

ener

ate

cand

idat

es, a

nd u

se B

itmap

of

Dp

to c

heck

freq

uenc

y su

ppor

t in

Dp.

c

d ee

f

g

1

d b f hac d e bZB

DD

(Dn)

Bitm

apa

b c

d e

f g h

iP

1: 1

0 0

0 1

0 1

0 0

P2:

1 0

0 1

0 0

0 0

1P

3: 0

1 0

0 0

1 0

1 0

P4:

0 0

1 0

1 0

0 1

0

N1:

1 0

0 0

0 1

1 0

0N

2: 0

1 0

1 0

0 0

1 0

N3:

0 1

0 0

0 1

0 1

0N

4: 0

0 1

0 1

0 1

0 0

Dp= Dn=

Ord

erin

g: a

<c<d

<e<b

<f<g

<h

hf

bi

da

ge

a

eA2

hA3

cA1

hf

bh

db

gf

a

eA2

gA3

cA1

Dp

Dn

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g53

Con

tras

t pat

tern

bas

ed c

lass

ifica

tion

--hi

stor

yC

ontr

ast p

atte

rn b

ased

cla

ssifi

catio

n: M

etho

ds to

bui

ld o

r im

prov

e cl

assi

fiers

, usi

ng c

ontr

ast p

atte

rns

CB

A (

Liu

et a

l 98)

CA

EP

(Don

g et

al 9

9)In

stan

ce b

ased

met

hod:

DeE

Ps

(Li e

t al 0

0, 0

4)Ju

mpi

ng E

P b

ased

(Li

et a

l 00)

, Inf

orm

atio

n ba

sed

(Zha

ng e

t al 0

0), B

ayes

ian

base

d (F

an+K

otag

iri03

), im

prov

ing

scor

ing

for

>=

3 cl

asse

s (B

aile

y et

al 0

3)

CM

AR

(Li

et a

l 01)

Top

-ran

ked

EP

bas

ed P

CL

(Li+

Won

g 02

)C

PA

R (

Yin

+H

an 0

3)W

eigh

ted

deci

sion

tree

(Alh

amm

ady+

Kot

agiri

06)

Rar

e cl

ass

clas

sific

atio

n(A

lham

mad

y+K

otag

iri04

)C

onst

ruct

ing

supp

lem

enta

ry tr

aini

ng in

stan

ces

(Alh

amm

ady+

Kot

agiri

05)

Noi

se to

lera

nt c

lass

ifica

tion

(Fan

+Kot

agiri

04)

EP

leng

th b

ased

1-c

lass

cla

ssifi

catio

n of

rar

e ca

ses

(Che

n+D

ong

06)

…M

ost f

ollo

w th

e ag

greg

atin

g ap

proa

ch o

f CA

EP

.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g54

EP

-bas

ed c

lass

ifier

s: r

atio

nale

Con

side

r a

typi

cal E

P in

the

Mus

hroo

m d

atas

et, {

odor

= n

one,

st

alk-

surf

ace-

belo

w-r

ing

= s

moo

th, r

ing-

num

ber

= o

ne};

its s

uppo

rt

incr

ease

s fr

om 0

.2%

from

“po

ison

ous”

to 5

7.6%

in “

edib

le”

(gro

wth

ra

te =

288

).

Str

ong

diffe

rent

iatin

g po

wer

: if a

test

T c

onta

ins

this

EP

, we

can

pred

ict T

as

edib

le w

ith h

igh

conf

iden

ce 9

9.6%

= 5

7.6/

(57.

6+0.

2)A

sin

gle

EP

is u

sual

ly s

harp

in te

lling

the

clas

s of

a s

mal

l fra

ctio

n (e

.g. 3

%)

of a

ll in

stan

ces.

Nee

d to

agg

rega

teth

e po

wer

of m

any

EP

s to

mak

e th

e cl

assi

ficat

ion.

E

P b

ased

cla

ssifi

catio

n m

etho

ds o

ften

out p

erfo

rm s

tate

of t

he a

rt

clas

sifie

rs, i

nclu

ding

C4.

5 an

d S

VM

. The

y ar

e al

so n

oise

tole

rant

.

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g55

CA

EP

(C

lass

ifica

tion

by A

ggre

gatin

g E

mer

ging

Pat

tern

s)

The

cont

ribut

ion

of o

ne E

P X

(sup

port

wei

ghte

d co

nfid

ence

):

Giv

en a

test

T a

nd a

set

E(C

i) of

EPs

for c

lass

Ci,

the

aggr

egat

e sc

ore

of T

for C

i is

Giv

en a

test

cas

e T,

obt

ain

T’s

scor

es fo

r eac

h cl

ass,

by

aggr

egat

ing

the

disc

rimin

atin

g po

wer

of E

Ps c

onta

ined

by

T; a

ssig

n th

e cl

ass

with

the

max

imal

sco

re a

s T’

s cl

ass.

The

disc

rimin

atin

g po

wer

of E

Ps a

re e

xpre

ssed

in te

rms

of

supp

orts

and

gro

wth

rat

es. P

refe

rla

rge

supR

atio

, lar

ge s

uppo

rt

For

eac

h cl

ass,

usi

ng m

edia

n (o

r 85

%)

aggr

egat

ed v

alue

to

norm

aliz

e to

avo

id b

ias

tow

ards

cla

ss w

ith m

ore

EP

s

Com

pare

CM

AR

: C

hi2

wei

ghte

d C

hi2

stre

ngth

(X)

= s

up(X

) *

supR

atio

(X)

/ (su

pRat

io(X

)+1)

scor

e(T,

Ci)

=Σ

stre

ngth

(X)

(ove

r X

of C

imat

chin

g T

)

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g56

How

CA

EP

wor

ks?

An

exam

ple

Giv

en a

test

T=

{a,d

,e},

how

to c

lass

ify T

?

b

ed

cb

ea

ed

ca

ed

ba

ec

dc

ba

baC

lass

2 (D

2)

Cla

ss 1

(D1)

●T

cont

ains

EPs

of c

lass

1 :

{a,e

} (50

%:2

5%) a

nd

{d,e

} (50

%:2

5%),

so

Sco

re(T

, cla

ss1)

=

●T

cont

ains

EPs

of c

lass

2: {

a,d}

(25%

:50%

), so

Sc

ore(

T, c

lass

2) =

0.3

3;

●T

will

be c

lass

ified

as

clas

s 1

sinc

e Sc

ore1

>Sco

re2

0.5*

[2/(

2+1)

] + 0

.5*[

2/(2

+1)

] = 0

.67

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g57

DeE

Ps

(Dec

isio

n-m

akin

g by

Em

ergi

ng P

atte

rns)

An

inst

ance

bas

ed (

lazy

)le

arni

ng m

etho

d, li

ke k

-NN

; but

doe

s no

t us

e no

rmal

dis

tanc

e m

easu

re.

For

a te

st in

stan

ce T

, DeE

Ps

Firs

t pro

ject

eac

h tr

aini

ng in

stan

ce to

con

tain

onl

y ite

ms

in T

Dis

cove

r E

Ps

from

the

proj

ecte

d da

taT

hen

use

thes

e E

Ps

to s

elec

t tra

inin

g da

ta th

at m

atch

som

e di

scov

ered

E

Ps

Fin

ally

, use

the

prop

ortio

nal s

ize

of m

atch

ing

data

in a

cla

ss C

as T

’s

scor

e fo

r C

Adv

anta

ge: d

isal

low

sim

ilar

EP

s to

giv

e du

plic

ate

vote

s!

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g58

DeE

Ps

: Pla

y-G

olf e

xam

ple

(dat

a pr

ojec

tion)

Tes

t =

{su

nn

y, m

ild, h

igh

, tru

e} Out

look

Tem

pera

ture

Hum

idity

Win

dyC

lass

sunn

yhi

ghN

sunn

yhi

ghtru

eN

true

Nsu

nny

mild

high

Nm

ildhi

ghtru

eN

high

Pm

ildhi

ghP

TRU

EP

sunn

yP

mild

Psu

nny

mild

TRU

EP

mild

high

TRU

EP

Out

look

Tem

pera

ture

Hum

idity

Win

dyC

lass

sunn

yho

thi

ghfa

lse

Nsu

nny

hot

high

true

Nra

inco

olno

rmal

true

Nsu

nny

mild

high

fals

eN

rain

mild

high

true

Nov

erca

stho

thi

ghFA

LSE

Pra

inm

ildhi

ghFA

LSE

Pra

inco

olno

rmal

FALS

EP

over

cast

cool

norm

alTR

UE

Psu

nny

cool

norm

alFA

LSE

Pra

inm

ildno

rmal

FALS

EP

sunn

ym

ildno

rmal

TRU

EP

over

cast

mild

high

TRU

EP

over

cast

hot

norm

alFA

LSE

P

Dis

cove

r E

Ps

and

deriv

e sc

ores

usi

ng th

e pr

ojec

ted

data

Orig

inal

dat

aP

roje

cted

dat

a

IEE

E IC

DM

28-

31 O

ct. 0

7C

ontr

ast D

ata

Min

ing:

Met

hods

and

App

licat

ions

Ja

mes

Bai

ley

and

Guo

zhu

Don

g59

PC

L (P

redi

ctio

n by

Col

lect

ive

Like

lihoo

d)Le

t X1,

…,X

mbe

the

m (

e.g.

100

0) m

ost g

ener

al E

Ps

in d

esce

ndin

g su

ppor

t ord

er.

Giv

en a

test

cas

e T

, con

side

r th

e lis

t of a

ll E

Ps

that

mat

ch T

. Div

ide

this

list

by

EP

’s c

lass

, and

list

them

in d

esce

ndin

g su

ppor

t ord

er:

P c

lass

: Xi1

, …, X

ip

N c

lass

: Xj1

, …, X

jn

Use

k (

e.g.

15)

top

rank

ed m

atch

ing

EP

s to

get

sco

re fo

r T

for

the

P

clas

s (s

imila

rly fo

r N

):

norm

aliz

ing

fact

or

Sco

re(T

,P)

= Σ

t=1k

supp

P(X

it) /

supp

P(X