PCP Quality Assessment Protocol

The Preprocessed Connectomes Project

Quality Assessment Protocol - a resource for measuring the

quality of MRI data

Cameron CraddockComputational Neuroimaging Lab

Center for Biomedical Imaging and NeuromodulationNathan S. Kline Institute for Psychiatric Research

Center for the Developing BrainChild Mind Institute

Neuroimaging meets big data

1,112 rs-fMRI and MRI datasets539 w/ASD, 573 Typical

1,629 Healthy Controls3,357 MRI scans

5,093 rs-fMRI scans

Sharing preprocessed data

• Make data available to a wider audience of researchers

• Evaluate reproducibility of analysis results

http://preprocessed-connectomes-project.github.io/

Which of the data do you include in an analysis?

• Use it all and hope that the introduced error will be swamped out by the large number of samples? – data quality vs. size trade-off

• Only use the best data as determined by visual inspection?– Limited by intra- and inter- rater reliability– Is the human eye good enough to identify data that is good

enough? (what is “good enough” for data?)• Choose data based on quantitative metrics?

– Which Metrics?– What thresholds?

Motion Crisis

Motion Crisis 2016

Manualized manual inspection

https://github.com/SIMEXP/niak_manual/blob/master/qc_manual_v1.0/qc_manual_niak.pdf

Quality Assessment Protocol

http://preprocessed-connectomes-project.github.io/quality-assessment-protocol/

Quality Assessment Protocol (2)- Open source pipeline for calculating quality assessment measures

from raw MRI data- Implemented in Python using Nipype

- Fast execution on cluster and multi-core computers- Restart processing without having to redo everything

- Uses AFNI and custom Python functions- Runs on Mac OSX, Linux, BSD, and Unix

Spatial Measures• sMRI & fMRI• Foreground to Background Energy

Ratio• Contrast to Noise Ratio (sMRI only)• Entropy Focus Criterion• Smoothness (FWHM)• % Artifact Voxels• Signal-to-Noise Ratio• Ghost-to-Signal Ratio (fMRI only)

Mortamet et al 2009

Spatial Measures• sMRI & fMRI• Foreground to Background Energy

Ratio• Contrast to Noise Ratio (sMRI only)• Entropy Focus Criterion• Smoothness (FWHM)• % Artifact Voxels• Signal-to-Noise Ratio• Ghost-to-Signal Ratio (fMRI only)

Temporal Measures • Global Correlation (GCOR)• Standardized DVARS• Median distance index• Mean Functional

Displacement• # Voxels with FD > 0.2m• % Voxels with FD > 0.2m

Reports

Repository of data measures

• Normative datasets to help learn thresholds for quality control– ABIDE– CoRR

http://preprocessed-connectomes-project.github.io/quality-assessment-protocol/

B

Correlation Between Measures

A

Correlation between measures for structural (A) and functional (B) data. ABIDE on lower triangle, CoRR on upper triangle. X values are insignificant at FDR corrected q < 0.05.

Most discriminative measures

Green is good data, red is poor data as determined by manual assessments from four expert raters (consensus).

Test-Retest Reliability

Acknowledgements• Steve Giavasis, MS @ Child Mind Institute – developer• Sang Han Lee, PhD @ Nathan Kline Institute – analysis• John Pellman, BA @ Child Mind Institute –

documentation and support• Zarrar Shehzad, MA @ Yale – initial implementation• Chris Gorgelewski, PD @ Stanford – contributor

06.26 - 06.30 OHBM Hackathon,

Lausanne09.18 - 09.20

Brainhack Vienna10.10 - 10.12

Brainhack LA