Program Update April 11, 2013 Andrew J. Buckler, MS Principal Investigator, QI-Bench

Program UpdateApril 11, 2013

Andrew J. Buckler, MSPrincipal Investigator,

QI-Bench

WITH FUNDING SUPPORT

PROVIDED BY NATIONAL

INSTITUTE OF STANDARDS AND

TECHNOLOGY

Agenda• Summary and close-out

of 1H FY13 reporting period:– Covering what’s been

accomplished from multiple points of view

• Progress since F2F:– QIBA Challenge support– Architectural components– Publications

22

Overall

Speci

fy

Form

ulate

Execu

te

Analyze

Package

Iterat

e

Studies

0

10

20

30

40

50

60

ResolvedOpen

Overall

Speci

fy

Form

ulate

Execu

te

Analyze

Package

Iterat

e

Studies

0

5

10

15

20

25

30

35

ResolvedOpen

3

Autumn 2012 (n=42)FY 2012 (n=110)

Winter 2013 (n=13) In Queue (n=23)

333

Overall

Speci

fy

Form

ulate

Execute

Analyze

Package

Iterat

e

Studies

0

2

4

6

8

10

12

14

ResolvedOpen

Overall

Speci

fy

Form

ulate

Execute

Analyze

Package

Iterat

e

Studies

0

1

2

3

4

5

6

ResolvedOpen

Overal

l

Speci

fy

Form

ulate

Execute

Analyze

Package

Iterat

e

Studies

0

2

4

6

8

10

12

ResolvedOpen

• Established initial structure• Initial Specify, Formulate,

and Iterate• Substantial work in Execute

and Analyze• First work on test-beds

• V&V issues in Execute• Substantial work in Analyze

library• Deployment support• Advanced test-beds

• Architectural design for next generation

• Expansion to contour-based analysis

• First comprehensive analysis library runs on vCT test-bed

• PACS interface to RDSM• Early work on triple store

• QI-Bench REST• First workstation support• Realization of QI-Bench

cohesive architecture• Migrate worked example

test-bed into triples

5555

Secondary AnalysisRegister all

seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them

within same acqrepFor each SUBJID and for each ACREP

STAPLE Weights.niiBinarize Weights-bin.nii

Pair-wise comparison between seg_SUBJID_0_ACREP_rdrID.nii-r.nii and

Weights-bin.nii

Create input files for analysis of metrics: Intersection, Union, Jaccard, DICE

Tertiary AnalysisRegister all

seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of themacross different acqrep

Primary Analysis

Prepare input files for statistical modules

Bland Altman, CCCLinear Mixed Effect

Probability Density Function of Error

(steps only needed to accommodate 1B)Segmentation Objects:

1.2…UID.dcm

Extract tags such as SUBJID, readerID, Acqrep

Rename toSeg_SUBJID_0_ACQREP_rdrID.dcm

Extract tags such as pixel spacing, matrix

Calculate volumes

Convert from dcm to nii format

Label map files from second 1B study

77777

Primary Analysis

Prepare input files for statistical modules

Bland Altman, CCCLinear Mixed Effect

Probability Density Function of Error

Files rdg_SUBJID_participantID.csvand seg_SUBJID_TPINDEX_participantID

from participants

Secondary AnalysisRegister all

seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them within same acqrep

For each SUBJID and for each ACREPSTAPLE Weights.nii

Binarize Weights-bin.niiPair-wise comparison between

seg_SUBJID_0_ACREP_rdrID.nii-r.nii and Weights-bin.nii

Create input files for analysis of metrics: Intersection, Union, Jaccard, DICE

Tertiary AnalysisRegister all

seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them across different acqrep

(steps only needed for 3A)

Disregard distractors

Re-assign TPINDEX to ACQREP

Roll-up to form s_files, including cov and dcm metadata

Supported segmentation object formats:• Analyze• mhd• mha• nrrd• nii

88888

QIBA Challenge Support

DEVELOPMENT PROGRESS REPORTwernsing

999

10

1111

1212

1313

1414

1515

Development plan

Continue with core development.1) Controller to support…2) QI-Bench API3) Connections to Computing Services4) Connections to Business Logic and Data Services

Parallel workCreating the initial Slicer plug-inRefining and augmenting the QI-Bench APIWorkflows and custom actors in Kepler

Connections to the APIWeb applications

1616

Recent Publications

1717

1818

Value proposition of QI-Bench• Efficiently collect and exploit evidence establishing

standards for optimized quantitative imaging:– Users want confidence in the read-outs– Pharma wants to use them as endpoints– Device/SW companies want to market products that produce them

without huge costs– Public wants to trust the decisions that they contribute to

• By providing a verification framework to develop precompetitive specifications and support test harnesses to curate and utilize reference data

• Doing so as an accessible and open resource facilitates collaboration among diverse stakeholders

1919

Summary:QI-Bench Contributions• We make it practical to increase the magnitude of data for increased

statistical significance. • We provide practical means to grapple with massive data sets.• We address the problem of efficient use of resources to assess limits of

generalizability. • We make formal specification accessible to diverse groups of experts that are

not skilled or interested in knowledge engineering. • We map both medical as well as technical domain expertise into

representations well suited to emerging capabilities of the semantic web. • We enable a mechanism to assess compliance with standards or

requirements within specific contexts for use.• We take a “toolbox” approach to statistical analysis. • We provide the capability in a manner which is accessible to varying levels of

collaborative models, from individual companies or institutions to larger consortia or public-private partnerships to fully open public access.

2020