Kernel Methods for De-noising with Neuroimaging Application Trine Julie Abrahamsen September 8, 2009.

Kernel Methods for De-noising withNeuroimaging Application

Trine Julie Abrahamsen

September 8, 2009

08/09/20092

Overview

• Introduction

• Kernel Principal Component Analysis

• The Pre-image Problem

• Analysis of Cimbi Data

• Conclusions

08/09/20093

Introduction

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data – Conclusions

• Kernel PCA for de-noising

• The pre-image problem is a key aspect in achieving good results

Objective

The over all aim of this project is two-fold

• To investigate the pre-image problem

• Apply kernel methods for de-noising on neuroimaging data

08/09/20094

Introducing Kernels

The idea of kernel methods

Often

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

The Gaussian kernel

Definition of kernel function:

Mika et al. 1999

Schölkopf et al. 2001

08/09/20095

Kernel Principal Component Analysis

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

Linear PCA is performed in feature space. Thus, the first PC can be found as the normal direction, v1 , by

All solutions must lie in the span of the training images, hence,

The projection of onto the i’th PC can be found as

While the projection onto the subspace spanned by the first q PCs is given by

Schölkopf et al. 1998

08/09/20096

Kernel PCA - Illustration

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

PCA

Kernel PCA

08/09/20097

The Pre-Image Problem

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

The pre-image problem = reconstruction of point in input space from feature space point

Ill-posed due to properties of the -map.

Relax search to find approximate pre-image

Common methods seek to minimize the feature space distance

where

Mika et al. 1999

Schölkopf et al. 1999

08/09/20098

Overview of Current Estimation Schemes

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

Mika et al.

Kwok & Tsang

Dambreville et al.

08/09/20099

Input Space Distance Regularization

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

Which is equivalent to minimizing

For RBF kernels the cost function (which should be maximized) reduces to

08/09/200910

Experiments on the USPS Data Set

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

USPS digits Gaussian noise added Mika et al. Input space regularization

Hull 1994

08/09/200911

Experiments on the USPS Data Set

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

(a) Kwok & Tsang (2004)

(b) Mika et al. (1999)

(c) Dambreville et al. (2006)

(d) Input Space Reg.

Evaluating the stability by confidence intervals on the mean squared error.

08/09/200912

Introducing the Cimbi Analysis

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

• De-noising for refining statistical significance

• D = 14 regions representing the frontolimbic area.

• Serotonin receptor binding potential quantified from

PET scans

• Neuroticism, Anxiety, and Vulnerability

• N = 129.

• Initial experiments on training and test sets

www.cimbi.org

Frøkjaer 2008

08/09/200913

Applying Kernel PCA De-noising

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

Vuln

era

bili

ty Original De-noised

Neuroticism p=0.11 p=0.057*

Anxiety p=0.063* p=0.015**

Vulnerability p=0.039** p=0.018**

08/09/200914

Learning curves

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

• 100 subsets sampled without replacement

• Data set sizes 5,10,…,125,129

• c chosen as 5th percentile and q chosen so 65% of variance is described

• Dambreville et al.

08/09/200915

Conclusions

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

• Adding input space distance regularization stabilizes the pre-image with limited sacrifice in terms of de-noising efficiency

• For the Cimbi data Dambreville et al.’s method proved very efficient

• When working on all 129 subjects a remarkable decrease in p-value could be achieved for both neuroticism, vulnerability, and anxiety

• Derive guidelines for choosing the regularization parameter λ

• Introduce other types of regularization

• Investigate other applications of kernel PCA (e.g., outlier detection)

• Improve performance by varying the kernel and its parameters

• Include non-linear adjustment for age and gender

Future Work

08/09/200916

References

Kernel Methods for De-noising withNeuroimaging Application

Trine Julie Abrahamsen

September 8, 2009

08/09/200918

Many local minima with almost equal value

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

08/09/200919

Distance distortions for non-linear kernels

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

08/09/200920

Permutation Test

Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions

• Non-parametric test

• Re-arrange trait score on BP

• p-value is found as the proportion of sampled permutations where the correlation is greater or equal to the correlation found on the original data.

08/09/200921

Experiments on the USPS Data Set

Introduction - Kernel PCA - The Pre-Image Problem - Analysis of Cimbi Data - Conclusions

08/09/200922

Bootstrap Resampling

Introduction - Kernel PCA - The Pre-Image Problem - Analysis of Cimbi Data - Conclusions

Log(p

) aft

er

de-n

ois

ing

Log(p) before de-noising