Top Banner
Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University
28

Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Mar 30, 2015

Download

Documents

Nichole Goffe
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Towards Robust Medical Image Segmentation

Shaoting ZhangCBIM Center

Computer Science DepartmentRutgers University

Page 2: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

IntroductionBackground

• Segmentation (finding 2D/3D region-of-interest) is a fundamental problem and bottleneck in many areas.

• We focus on learning-based deformable models with shape priors (2D contour or 3D mesh).

Chest X-ray

Lung CADLiver in whole-

body CT (PET-CT)

Rat brain structures in MR Microscopy

2

Page 3: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

IntroductionBackground

• End-to-end, automatic, accurate, efficient.

• Robustness– Handle weak or misleading

appearance cues.– Handle diseased cases (e.g.,

with tumor/cancer).– Leverage shape priors to

improve the robustness (Active Shape Model, T. Cootes, CVIU’95; 3D ASM for cardiac segmentation, Y. Zheng, TMI’08)

Segmentation system

3

Liver shape variations[Springer images]

Page 4: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

IntroductionResearch Void

• Limitations of existing shape prior methods:– Assume Gaussian errors → Sensitive to outliers– Assume unimodal distribution of shapes → Cannot

handle large shape variations, e.g., multimodal– Only keep major variation → Lose local shape detail

4

Page 5: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

IntroductionResearch Void

• Handling gross errors or outliers. – RANSAC + ASM [M. Rogers, ECCV’02]– Robust Point Matching [J. Nahed, MICCAI’06]

• Handling multimodal distribution of shapes.– Mixture of Gaussians [T. F. Cootes, IVC’97]– Manifold learning for shape prior [Etyngier, ICCV’07]– Patient-specific shape [Y. Zhu, TMI’10]

• Preserving local shape details.– Sparse PCA [K. Sjostrand, TMI’07]– Hierarchical ASM [D. Shen, TMI’03]

5

Need to solve all three

challenges simultaneously

in practice

Page 6: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsSegmentation framework

6

Landmark Detectors

Offline Learning

Boundary Detectors

Runtime Segmentation

New Image

Model Initialize

Iterative Deformation and Shape Refinement

Final Results

Shape Priors via Sparse Shape Representation

Image Data and Manually Labeled Ground Truths

Critical anatomical landmarks.

Page 7: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsSegmentation framework

• Initialization: Landmark detectors + shape prior– Automatic, fast (AdaBoosting/PBT/random forest + 3D Haar).

• Deformation: Boundary detectors + shape prior– Extend the landmark detectors to locate the sub-surfaces.

Critical anatomical landmarks.

7

Lung Heart

Rib

Colon

Abdomen

7

Page 8: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsShape prior using sparse shape representation

• Our shape prior is based on two observations:– An input shape can be approximately represented by a

sparse linear combination of training shapes.– The given shape information may contain gross errors, but

such errors are often sparse....

8

Page 9: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsShape prior using sparse shape representation

• Formulation:–

• Sparse linear combination:–

...≈

...≈

Dense xSparse xAligned shape data matrix DWeight x

Input y

Global transformationparameter

Number of nonzero elements

Global transformationoperator

9

Page 10: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsShape prior using sparse shape representation

• Non-Gaussian errors:–

10

Page 11: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsShape prior using sparse shape representation

• Why it works?– Robust: Explicitly modeling “e” with L0 norm constraint.

Thus it can detect gross (sparse) errors, i.e., non-Gaussian– General: No assumption of any parametric distribution

model (e.g., a unimodal distribution assumption in ASM). Thus it can model large shape variations.

– Lossless: It uses all training shapes. Thus it is able to recover detail information even if the detail is not statistically significant in training data.

11

Page 12: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsOptimization

• Convex relaxation [Candes, Tao, T-IT’06]:

• Tuning parameters (λ1 and λ2) are not difficult to choose, compared to k1 and k2. – λ1 x║ ║1, λ1 controls the sparsity of x. – λ2 e║ ║1, λ2 controls the sparsity of e.

• One group of parameters work well for all images.• Efficient convex optimization: fast proximal gradient

methods (e.g., FISTA)12Shaoting Zhang

Page 13: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

MethodsScalability

13Shaoting Zhang

... ≈... ...

Dictionary size = 64 #Coefficients = 1000 #Input samples = 1000

1. To efficiently handle many training samples - dictionary learning:

2. To efficiently handle thousands of vertices - mesh partitioning:

Page 14: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part I 2D lung localization in X-ray

(Lung computer-aided diagnosis system, Siemens)• Handling gross errors

Detection PA ASM RASM NN TPS Sparse1 Sparse2

Procrustes analysisActive Shape ModelRobust ASMNearest NeighborsThin-plate-splineWithout modeling “e”

Proposed method

14

Page 15: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part I2D lung localization in X-ray

• Multimodal shape distribution

Detection PA ASM/RASM NN TPS Sparse1 Sparse2

15

Page 16: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part I2D lung localization in X-ray

• Recover local detail information

Detection PA ASM/RASM NN TPS Sparse1 Sparse2

16

Page 17: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part I2D lung localization in X-ray

• Sparse shape components

• ASM modes:

17

0.5760

+ +

0.2156 0.0982

Page 18: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part I2D lung localization in X-ray

• Mean values and standard deviations. ~1,000 cases.

Left lung

Right lung

1)PA, 2)ASM, 3)RASM, 4)NN, 5)TPS, 6)Sparse1, 7)Sparse2

µσ

18

Page 19: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

19

• Use case: Improved interpretation of PET-CT, with Siemens– PET: Measure functional process (metabolic activity), oncology– CT: Provide anatomical information, co-registered with PET– Issues: High variations of activities across different organs– Solution: Segment organs in CT for organ-specific interpret.– Challenge: Low dose results in low contrast & fuzzy boundary

Page 20: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

Procrustesanalysis Sparse shape Ground truth

Initialization

Deformation

20

Same landmarks + different shape priors

Same deformation module

Page 21: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

Procrustesanalysis

Sparse shape

21

Page 22: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

Procrustesanalysis

Sparse shape

22

Page 23: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

Procrustesanalysis

Sparse shape

23

Page 24: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

Procrustesanalysis

Sparse shape

24

Page 25: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

ASM-type [Zhan’09] Sparse shape Ground truth

• Shape refinement during segmentation

25

Page 26: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part II3D liver segmentation in low-dose CT

• Quantitative results: surface distances. ~80 cases

Mean value and standard deviation (voxel)

26

Page 27: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Applications – Part IIISegmentation of Multiple Brain Structures in MRI

Rat brain structures Drug addiction and alcoholism

(with Brookhaven National Lab)

Human brain, 34 structures Alzheimer's disease

(with GE Global Research)

27

Page 28: Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department Rutgers University.

Summary of Robust Segmentation• Robustly handle abnormal cases, such as diseased cases

(liver tumor). Critical to healthcare applications such as computational diagnosis systems.

• Patent with Siemens. Used in several clinical applications. Key contribution for our awarded NSF-MRI grant (’12-’16).

• Relevant publications:– First author papers

• MICCAI 2012, 2011 (MICCAI Young Scientist Award Finalist)• CVPR 2011• Two journal papers in Medical Image Analysis

– Second author paper• Medical Physics 2013 (with my co-mentored student, G. Wang)• ISBI 2013, oral (with my co-mentored student, Z. Yan)28