Ondřej Rozinek Czech Technical University in Prague Faculty of Biomedical Engineering 3D Hand Movement Analysis in Parkinson’s Disease 14.5.2008 1.

Ondřej RozinekCzech Technical University in Prague

Faculty of Biomedical Engineering

3D Hand Movement Analysis in Parkinson’s Disease

14.5.20081

Outline

Motivation and goalsColor calibrationMarker detectionCamera calibration and 3D reconstructionMovement analysis

Conclusion

Block diagram

14.5.20082

Motivation and goalsTask: Are there any changes in patient‘s conditions

after a drug was administered?

Solution: 3D video analysis of hand movement

3100 200 300 400 500 600 700

50

100

150

200

250

300

350

400

450

500

550

100 200 300 400 500 600 700

50

100

150

200

250

300

350

400

450

500

550

2D trajectory from top view

2D trajectory from side view

-300-200

-1000

100200

300-60

-40

-20

0

20

40

60

80

810

820

830

840

850

860

870

880

890

900

3D trajectory

Color calibration Correction of the image and so compensate different contrast

and brightness conditions

Task of curve fitting

Different color calibration methods are compared:

1. Linear interpolation (LI)2. Cubic Hermite functions (HF) 3. Multiple linear regression model (MLR)

Uncalibrated UncalibratedCalibrated Calibrated14.5.20084

Color calibration – multiple linear regression model

Let Y be the matrix of reference colors (image I) and X the corresponding colors of uncalibrated image J

t - number of terms MLR (linear combination of color components)n - used colors for color calibration t ≤ n - condition

Disadvantage: multicollinearity of colors: white, grayscale, black

JXXXYI Tntnt

Tntnt

1)(

Blu

e

Red Green

3D transfer function with linear terms

3D transfer function with non-linear terms

Blu

e

Red Green5

Color calibration - evaluation

Used color for calibration

c

Ecal

1D transfer functions 3D transfer function MLR(t)LI HF MLR3 MLR7 MLR10 MLR13

K, W 2 40.4 43.6 x x x xR, G, B 3 44.1 41.2 33.6 x x xC, M, Y 3 52.0 57.4 34.0 x x xC, M, Y, K 4 49.0 54.2 33.8 x x xR, G, B, K, W 5 34.0 40.8 33.0 x x xC, M, Y, K, W 5 47.3 52.9 31.5 x x xall 24 colored squares 24 27.3 31.0 29.1 22.8 16.8 15.1without calibration 159.8

n

irefirefirefical BBGGRR

nE

1

2221

black (K), white (W), red (R), green (G), blue (B), cyan (C), magenta (M), yellow (Y), c – number of corresponding colors, t –terms, t ≤ n 14.5.2008

6

Root mean square error: - reference values - calibrated values

refrefref BGR ,,

iii BGR ,,n - all squares on the color

chessboard

Marker detection1.2 seconds; 30 frames 2.0 seconds; 50 frames

14.5.20087

Camera calibration and 3D reconstruction

Pinhole camera model

- image coordinates - world coordinates - camera calibration matrix with intrinsic camera parameters - extrinsic camera parameters

Estimate the camera matrix Direct linear estimation Closed-form solution

Estimate the fundamental matrix relationship between the locations of two cameras using eight point alghoritm for point correspondences (u, v) for m ≥ 8 (i = 1,…m)

XXXx MRR

ofS

ofSfSRR

IKT

TT

yy

xx

T

TT

1

t

100

01

t0 1333

00

KXx

M

0

m

i

v

v

u

u

FTm

TiF

,R tChessboard for point correspondences

14.5.20088

Camera calibration and 3D reconstruction

Barrel distortion

dyrararayy

dxrararaxx

d

d

6

34

22

1

63

42

21

1

1

Undistorted

.22

222

45

2254

ddd

ddd

yrayxady

xrayxadx

For measurements is necessery undistorted image

- distorted image coordinates

- tangential distortion

- camera parameters

- new normalized point coordinate

dd yx ,

kadxdy,

yx, 22dd yxr

9

Movement analysis

-250 -200 -150 -100 -50 0 50 100 150 200 250

-60

-40

-20

0

20

40

60

80

810

820

830

840

850

860

870

880

890

900

1

2

3

4

136.5°

1

2

3

4

119.4°

14.5.200810

Movement analysis

Motion a b c d e

S 1.47

2.89 6.53 5.94 3.12

V 0.39

0.66

0.85 0.81 0.50

Rvar 5.50

9.85 22.21 17.57

9.34

Sk0.19

1.04

1.02 0.61 0.09

Ek2.10

2.86 2.63 1.78 1.58

standart deviation (S)

variation coefficient (V)

range (Rvar)

skewness (Sk)

kurtosis (Ek)

14.5.200811

Conclusion

14.5.200812

Blue markers are proposed

3D hand trajectory of patients is obtained

Error is 1-3 mm at rest and for slower motion (camera has only 25 frames per second)

Color calibration to obtain the required brightness and contrast for the segmentation

Hand velocity, angle in wrist and some statistic parameters are evaluated

Future plans

Thank you for your attention

14.5.200813