A man-machine human interface for a special device of the pervasive computing world

A man-machine human A man-machine human interface for a special interface for a special

device of the pervasive device of the pervasive computing worldcomputing world

B. Apolloni, S. Bassis, A. Brega,B. Apolloni, S. Bassis, A. Brega,

S. Gaito, D. Malchiodi, A.M. ZanaboniS. Gaito, D. Malchiodi, A.M. Zanaboni

DSI - University of Milano (I)DSI - University of Milano (I)

OutlineOutline

A procedure detecting attention states in car A procedure detecting attention states in car drivingdriving

Fed by biologic input supplied through non Fed by biologic input supplied through non invasive sensorsinvasive sensors

Explains its output through a possibly Explains its output through a possibly interpretable ruleinterpretable rule

The dataThe data

One subject using a car driver simulatorOne subject using a car driver simulator Subjected to alternate attention demanding Subjected to alternate attention demanding

manoeuvres (fast lane exchange, pedestrian manoeuvres (fast lane exchange, pedestrian avoidance) and relaxed drivingavoidance) and relaxed driving

4 signals traced by a Biopac device (SKT, 4 signals traced by a Biopac device (SKT, GSR, ECG, RSP)GSR, ECG, RSP)

Collected by the School of Psychology, Collected by the School of Psychology, Queen’s University Belfast.Queen’s University Belfast.

PreprocessingPreprocessing

Extracted featuresExtracted features 8 conventional (from medical knowledge)8 conventional (from medical knowledge) FFT processing for ECGFFT processing for ECG Drift of the ECG signal from a neural predictionDrift of the ECG signal from a neural prediction SKT not considered (constant)SKT not considered (constant)

Feature processing IFeature processing I

15 Boolean values 15 Boolean values are extracted from a are extracted from a time-window of time-window of width 3width 3

t-1 t+1t

€

Ep = log zpj−zpj

j=1

n

∏ (1− zpj )−(1−zpj )

⎛

⎝ ⎜

⎞

⎠ ⎟

Result of a Boolean Result of a Boolean ICA through ICA through minimization of minimization of empirical entropyempirical entropy

253 connections (after pruning)253 connections (after pruning)

Feature processing IIFeature processing II

Boolean values interpreted as propositional Boolean values interpreted as propositional variablesvariables

Minimal DNF and DNF on variables Minimal DNF and DNF on variables interpreted as symbolic waveletsinterpreted as symbolic wavelets

Begin

DNF=ø;

for each positive example u; DNF = DNF {m};return DNF;End

An obtained CNFAn obtained CNF(x1+x3+x6+x8+x10+x14)(x1+x2+x5+x6+x7+x9+x11+x12+x13)(x1+x2+x6+x7+x9+x10+x12+x14)(x1+x2+x5+x7+x8+x12+x13+x15)(x1+x3+x5+x11+x13+x15)(x3+x8+x10+x11+x13+x14+x15)(x1+x2+x4+x6+x7+x11+x12+x13+x14+x15)(x1+x2+x4+x8+x12+x13+x14+x15)(x1+x2+x3+x7+x8+x10+x13+x14+x15)(x3+x4+x8+x10+x13+x14+x15)(x1+x3+x6+x7+x8+x13+x14)(x3+x5+x7+x8+x13)(x2+x3+x6+x7+x10+x11+x12+x13+x14)(x1+x5+x6+x8+x9+x11+x12+x15)(x1+x5+x6+x8+x9+x11+x14+x15)(x3+x5+x7+x9+x10+x11)(x1+x3+x4+x7+x13+x14+x15)(x1+x2+x3+x8+x11+x13+x14+x15)(x2+x5+x6+x7+x9+x11+x12+x15)(x2+x3+x4+x6+x9+x10+x11+x12+x14)(x2+x4+x5+x7+x8+x13+x15)(x3+x4+x6+x7+x10+x13)(x3+x4+x6+x8+x10+x14)(x1+x4+x7+x8+x13+x15)(x1+x4+x5+x7+x8+x9+x15)(x4+x7+x8+x10+x15)(x4+x7+x8+x10+x14)(x2+x4+x5+x6+x7+x9+x10+x12+x13+x15)(x1+x3+x5+x8+x13+x15)(x1+x3+x6+x9+x10+x11+x14)(x1+x6+x9+x10+x11+x12+x14)(x1+x6+x8+x9+x11+x12+x14)(x1+x2+x4+x6+x7+x10+x11+x12+x13+x14)(x2+x4+x5+x8+x9+x10+x12+x13+x15)(x2+x4+x5+x7+x9+x10+x11+x12)(x1+x3+x6+x7+x8+x10+x13)(x3+x4+x5+x6+x8+x9+x10+x11)(x2+x5+x6+x8+x9+x10+x11+x12)(x2+x5+x6+x7+x8+x9+x11+x12)(x1+x2+x4+x5+x6+x7+x9+x12+x13)(x3+x4+x5+x7+x10+x11)(x1+x2+x5+x8+x10+x12+x13+x15)(x1+x2+x6+x7+x9+x10+x11)(x1+x2+x5+x7+x8+x9+x14)(x3+x4+x5+x8+x9+x10+x13+x15)(x3+x8+x9+x10+x11+x13)(x3+x9+x10+x11+x13+x14)(x1+x6+x7+x8+x9+x10+x12)(x1+x2+x3+x6+x11+x13+x14+x15)(x2+x4+x5+x6+x9+x11+x12+x14+x15)(x2+x6+x7+x10+x11+x12+x13+x15)(x2+x4+x6+x10+x11+x12+x14+x15)(x2+x5+x6+x9+x10+x11+x12+x14+x15)(x3+x4+x6+x8+x11+x14+x15)

Post processingPost processing

Simplification of the learnt rules through Simplification of the learnt rules through stochastic optimization of the coststochastic optimization of the cost

€

O f ,λ( ) = λ1 Lii=1

m

∑ + λ2 ρ +i

i=1

m

∑ + λ3 ρ −i

i=1

m

∑ + λ4ν 0

L: rule length, L: rule length, :rule radius, :rule radius, :disregarded :disregarded pointspoints

A simplified CNFA simplified CNF

(x6+x11+x1+x13)(x10+x12+x9+x6)(x1+x13+x11+x5)(x3+x8+x6)(x4+x1+x13)(x12+x6+x7+x13)(x13+x8+x9+x4)(x1+x6+x8)(x12+x6+x7+x8)(x1+x8+x5+x7)(x4+x6+x7+x13)(x7+x9+x10+x11+x3)(x1+x8+x13+x15)(x3+x8+x13+x15)(x4+x7+x8)(x1+x6+x9+x10+x11)(x2+x6+x11+x12+x15)(x3+x5+x8+x13)(x4+x5+x7+x10+x11)(x3+x9+x10+x11+x13)

From 403 to 81 literalsFrom 403 to 81 literals

Performance IPerformance I

DNFDNF CNFCNF

LengthLength FPFP FNFN LengthLength FPFP FNFN

AVGAVG 44.4244.42 22.9122.91 27.6227.62 71.0671.06 22.4122.41 28.2228.22

STDVSTDV 7.357.35 5.365.36 8.798.79 20.3720.37 7.497.49 5.445.44

50 cross-validation test50 cross-validation test FP: false positives; FN: false negativesFP: false positives; FN: false negatives

Performance IIPerformance II

LengthLength TPTP FPFP FNFN

DNFDNF 44.4244.42 72.3872.38 22.9122.91 27.6227.62

LengthLength TNTN FNFN FPFP

CNFCNF 71.0671.06 71.7871.78 22.4222.42 28.2228.22

Performance IIIPerformance III

Cones

-20

-10

0

10

20

30

40

50

60

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

Center lineSteeringAccelerationBrakeContinuos DNF resultIntegral

Pedestrian

-30

-20

-10

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Center lineSteeringAccelerationBrakeDNF frequency resultMajority continuity

Pedestrian

-30

-20

-10

0

10

20

30

40

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Center lineSteeringAccelerationBrakeContinuous DNF resultIntegral

Cones

-20

-10

0

10

20

30

40

50

60

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

Center lineSteeringAccelerationBrakeDNF frequency resultMajority Continuity