Top Banner
Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern University. Supported by DoDPI
37

Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Dec 16, 2015

Download

Documents

Jimmy Calvery
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Countermeasures to P300-based Guilty Knowledge Tests of Deception

J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary RobertsonNorthwestern University.

Supported by DoDPI

Page 2: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Some History (earliest publications)

Rosenfeld et al., 1987,1988,1991Farwell and Donchin, 1991Allen, Iacono, & Danielson, 1992Johnson and Rosenfeld, 1992Since we were there at beginning,

why do we challenge now with countermeasures? (1) It’s about time….

Page 3: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

2) Farwell’s web page, claiming 100% accuracy:

Page 4: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Stimuli:

Probes (P or R in figures): Items which subject is suspected of knowing (e.g., murder weapons). Subject denies(lies).

Targets (TR) Items: Items to which subject presses ‘YES’ . (Benchmark P300).

Irrelevants (I or W in figures): Items of which subject has no knowledge and denies, honestly, by pressing ‘NO’ .

Page 5: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

How P300 amplitude is supposed to catch Liars:

1)P>I (‘BAD’)2)P-TR corr>P-I corr(‘BC-AD’)

1)P=I2)P-I corr>P-TR corr

Page 6: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Whither R-TR correlation if there are latency differences?

Probe P3 Target P3

Nothing should happen to bootstrapped amplitude difference test (BAD) but bootstrapped cross-correlation test (BC-AD) should fail.

Page 7: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Experiment 1, based on Farwell & Donchin (1991):

--6 Different Probes --Innocent, Guilty, and Countermeasure(CM)

Groups--Countermeasure: Associate various latent

responses to different categories (jewelry type, drawer color, operation name, etc.), all irrelevant members of the category.

--”Off the Street “ subjects (Psych 101).

Page 8: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

General Instructions….

Mock crime scenarioPress “Yes” to Targets (on list)

Press “No” to all other stimuli (Possibly guilty probes and Irrelevants).

Page 9: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

More simply….

Probe Target I1 I2 I3 I4 ring bracelet necklace watch broach tiara

pink brown yellow purple red bluedonkey tiger lion cow pig horse etc., etc…... (only half the matrix here.)

All these are shuffled, presented in random order, involving 4 repetitions of each item.

Page 10: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

What are the covert countermeasures for the 6 categories of 6 probes?

1) Jewelry category……….micro right finger wiggle 2) drawer lining category…. “ left “ “ 3) owner’s name category…. “ “ toe “ 4) operation name category…” right “ “ 5) location of item category…… Imagine professor

slaps you 6) desktop category……………Do Nothing

I.e., make irrelevants into relevant targets.

Page 11: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Guilty group: Probe(R) > Irrelevant (W).

R > W

Page 12: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Guilty Group: TR vs R

Both have P300

Page 13: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Innocent Group: R vs W

Both lack P300

Page 14: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Innocent Group: TR vs R

TR towersover P (R)

Page 15: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

CM Group : R vs W

No differenceP(R) vs I (W)

Page 16: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

CM Group: Tr vs R

Target > Probe

Page 17: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Results, Exp. 1: CM works, and analysis method matters:

Diagnoses of Guilty

Guilty Group Innocent Group CM Group

9/11(82%) 1/11(9%) 2/11(18%)

Amplitude Difference (BAD) method,p=.1

Cross-Correlation(BC-AD) Method, p=.1

6/11(54%) 0/11(0%) 6/11(54%)

--with “Off the street” subjects; see next 2 slides.

Page 18: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Guilty Subject, probe(R) vs irrelevant(W)

R >> W: subjectclearly guilty

Page 19: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

…but clear latency shift in TR and R P300s

..so BC-AD fails,BAD catches the S.

Page 20: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Do Reaction Times help?

Guilty Group CM Group

9/1090% BAD

3/10*30% BART *

These are guilty calls based on bootstrapped(Probe-Irrelevant) RTs, at p =.05

.* Bootstrapped Analysis of RT difference, R-W.

Page 21: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Absolute Irrel. RTs for CM and Inno. Groups overlap*.

..and 3 fastest CM-ers beat tes,3 slowest did not

*I.e., they don’t helpeither.

Page 22: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Experiment 2: The one probe protocol (Rosenfeld et al., 1991).

100% simpler to conceptualize:There are 6 items, repeated 30-40

times randomly, consisting of..A probe, subject maybe guilty,A target to force attention (not really

necessary but allows us to test Farwell cross-correlation method vs. ours.)

4 irrelevants.

Page 23: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Standard autobiographical oddball paradigm.

6 dates are presented, one is the birth date.

5 other dates with no personal significance, one is a target, four are irrelevant.

Subject is feigning head injury, the inability to recognize birth date 50% of the time, so he presses yes and no to all dates on 50-50 basis.

Page 24: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Design:

One group of advanced Ss run in 3 successive weeks. (Also Control group.)

Week 1: All naïve(about CM) and guilty (of having birth date and recognizing it despite overt lies.

Week 2: Use explicit countermeasure (to be explained).

Week 3: Repeat first week, stop the CM.

Page 25: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

What’s the CM?

When you see the first Irrelevant, do micro left finger wiggle.

When you see second Irrelevant, do micro right toe wiggle.

When you see third irrelevant, imagine me slapping your face.

When you see fourth Irrelevant, do nothing.

I.e., make irrels. into rel. targets.

Page 26: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Results, week 1(guilty): Probe(R) vs.Irrelevant (W)

As usual, R>>W

Page 27: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Week 1: Probe(R) and Target(TR):

Both have nice P3

Page 28: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Week 2: Explicit CM, R v W

Not so differentanymore...

Page 29: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Week 3: 5/12 test beaters(effortless), R v W:

Surprise! R = W

Page 30: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Week 3: 5/12 test beaters, R v TR--”classic defeats”:

..and TR >> Ras with innocents.

Page 31: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Exp. 2; quantitative.

WK BAD BC-AD

no CM 12/13(.92) 9/13(.69) CM 6/12(.50) 3/12(.25) no CM 7/12(.58) 3/12(.25)

(Control group: nothing much happened over 3 weeks of repeating week 1.)

Page 32: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

RTs for 3 weeks; week 1= week 3, proving CM not used in week 3.

0 1 2 3 4WEEK

400

500

600

700

800

900

1000

1100

React

ion T

ime

IRRELEVTARGETPROBE

Page 33: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Irrelevant RTs, with and without CM. No overlap!

21

WEEKGROUP

0

500

1000

1500

2000D

WW

RT

051015Count

0 5 10 15Count

Page 34: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

But in week 3, RT no help, as we saw...

31

GROUP

300

400

500

600

700

800

900

1000D

WW

RT

05101520Count

0 5 10 15 20Count

Page 35: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Overall amplitude effects...

0 1 2 3 4WEEK

200

300

400

500

600

700A

MP

LIT

UD

E,p

-p,

com

pute

r unit s

.

IRRELEVTARGETPROBE

Page 36: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

Conclusions, bottom lines..

6-probe protocol beat-able, RT is no help, and the 6 probe combination lacks a real rationale anyway. (Lykken wouldn’t like?)

1-probe protocol may be explicitly beat-able, but the very slow Irrelevant RT distribution will raise suspicions. 1 probe per run is more Lykkenable.

BUT---1-probe paradigm after CM practice is beat-able, period.

Page 37: Countermeasures to P300- based Guilty Knowledge Tests of Deception J.Peter Rosenfeld, Matt Soskins,Joanna Blackburn, & Ann Mary Robertson Northwestern.

What to do?

We have found(submitted) that within individuals, the scaled scalp distribution method detects 73% (not great) using statistical criteria yielding 0 % false positives.

This method should be worked on, because there is no obvious CM as there is with simple amplitude.