Department of Computer Science Department of Computer Science Computer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.de http://cvpr.uni-muenster.de IAPR Workshop on Graph-based Representations in Pattern Recognition June 11 th -13 th , 2007 – Alicante GbR GbR ’07 ’07 Separation of the Retinal Vascular Graph in Arteries and Veins Speaker: Kai Rothaus Co-authors: P. Rhiem, X. Jiang CVPR Group, University of Münster Homepage: cvpr.uni- muenster.de
18
Embed
Department of Computer Science Computer Vision & Pattern Recognition Group IAPR Workshop on Graph-based Representations in.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
IAPR Workshop on Graph-based
Representations in Pattern Recognition
June 11th -13th, 2007 – Alicante (Spain)
GbR ’07GbR ’07
Separation of the Retinal Vascular Graph
in Arteries and Veins
Speaker: Kai RothausCo-authors: P. Rhiem, X. Jiang
CVPR Group, University of Münster
Homepage: cvpr.uni-muenster.de
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
22
Outline
Introduction
– Medical purpose
– Image-processing
Method
– SAT-problem specification (vessel labelling)
– Operations for graph manipulation (edge labelling)
– Solving Conflicts
Results
Conclusions and further work
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
33
Medical Purpose
Why retinal vessel are of interest?– Vessels of retina and brain are conjuct– Only on retina vessels are visible directly– Conclusions on diseases are possible
Anatomy of the eye– Vessels enter the eyeball at the optic disc– Vessels only branch (no reconnection)– Capillars are invisible
stronger central reflex poor central reflexnever crossing arteries never crossing veins
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
44
Vessel segmentation
Input: Retinal Image Output: Binary vessel image Many segmentation algorithms, based on
– Matched-filter– Tracking– Intensity riges or (1st moment deviations)– Curvature (2nd moment deviations)
Special difficulties– Handling of bifurcations and crossings– Central-light reflex– Different vessel width– Wide intensity spectrum– Pathological objects nearby
Mainly, we use hand-segmented images
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
55
Graph-based representation of the vasculature
Input: Binary vessel image Output: Vasculature graph1. Compute the skeleton of the vasculature2. Classify skeleton pixel in
– End pixel (form vertices of degree 1)– Connection pixel (form edges)– Branching pixel (form vertices of degree 3)– Crossing pixel (form vertices of degree 4)
– Segmentation errors could lead to small cycles– Discontinuous segmentation leads to an over-
fragmented graph representation– Skeleton of a crossing could lead to two branches
binary vessel image
skeleton image
vasculature graph
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
66
SAT-Problem Specification (vessel labelling)
Problem: Classify each vessel as artery (a) or vein (v) Mainly recent approaches are based on local features
– Colour, cross-profile, thickness, etc.– Work only good for thick vessels nearby the optic disc
We propose a structure-based approach (on vasculature graph)– Label each vessel segment vi as artery (Li = a) or vein (Li = v) – Formalise anatomical properties of the vasculature:
1. At branches only edges of the same labelling are involved2. At crossings an artery crossing a vein
– Construct logical clauses that describe the properties– Cumulate above rules for all vertices and formulate the SAT-problem– Solve this as a CSP (Constraint Search Problem) with AC-3
a a
a
v v
vv
v
a
aa
av
v
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
77
The labelling process (AC-3*)
1. Add the incident vertices of few manually labelled vessel segments in the process queue Q
2. While Q is not empty– Take the first vertex and corresponding logical rule– Reduce set of labels of the incident vessels consistent to the rule– If there is a conflict try to solve it (details later)– Otherwise add the new vertices to Q
Order of processing the vertices (rules) is important
conflict
manuallabel
conflict
manuallabel
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
99
Operations for graph manipulation (edge labelling)
Segmentation or skeleton errors lead to unsolvable SAT-problem
Graph structure has to be manipulated slightly Allowed operations should handle:
Instead of manipulating the graph directly we introduce a second order labelling (edge labelling):
vessel labelling
resolve problem label graph manipulationop1 1 c melt 2 branches to one crossingop2 2+3 e split a branchingop3 4 f delete an edge
– – n nothing (normal edge)
vasculature graph
edge labelling
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1010
Steering the labelling process (Belief propagation)
Plausibility weights [0,1] for each vertex– Assign crossing vertex the plausibility 1 - P1(d)
– Assign branch vertex the plausibility (with β = max αi) P1(d)+P2(β) - P1(d)P2(β)
Plausibility weights [0,1] for each a/v-labelled vessel – Assign hand-labelled vessels plausibility 1– During AC-3* algorithm use a multiplicative propagation
scheme (with weights of corresponding vertex and edge)
Use weights as heuristic to order Q as priority-queue
Use the average vessel weights to rate labelling results
P1(d)
P2(β)
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1111
Initial edge labelling
Decide on plausibility measures P1(d) and P2(β) if a connection edge between to branches is probably a crossing
No false c-label should be introduced Label edge with c-label iff [ d<3 ] or [ P1(d)<0.75 and P2(β)<P2(30°) ]
0
20
40
60
80
100
120
140
160
180
0 20 40 60 80 100 120
Max Angle
Seg
men
t L
eng
th
n-edges
c-edges
c-edges (missed)
C N
CGT 117 25
NGT 0 621
Confusion matrix on 10 training images
Accuracy of >96 %
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1212
Solving Conflicts
Conflicts cannot been avoided (even not with initial labelling) Conflicts are basically introduced by cycles in the vascular graph Topology is responsible for conflicts
Solving-strategy:– Search cycle (vertex set V’), where all vessel labels are defined– Establish edge candidate set E’={ e | e incident to a v in V’ }– Choose a “suitable” n-labelled edge of E’, with minimum weight and
change edge label to c (crossing)– Otherwise label the conflict edge with e (end-segment)– Restart the AC-3* algorithm
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1313
Interactive labelling tool
Requirement: binary vessel image
Physician mark single vessel segments as arteries an veins
Propagation of the manual labelling as far as possible
Solve logical conflicts automatically
If the result is not good enough for the observer, more vessel label
could be manually added
Presenting results in two different ways:artery (auto.) vene (auto.) artery (man.) vene (man.)
Original image
Binary image
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1414
Results on manually segmented images
STARE data set of A. Hoover et al. image im0082
manuel label init. c-label final c-label final e-label solved confl. avg. weight4 17 21 2 6/10 0.18
manuel label init. c-label final c-label final e-label solved confl. avg. weight6 17 21 3 7/9 0.21
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1515
Discussion results on manual segmentations
image manuel label init. c-label final c-label final e-label solved confl. avg. weight
0002 2 13 16 1 4/4 0.14
0003 4 7 8 0 1/1 0.24
0044 5 9 12 1 4/4 0.23
0077 5 7 15 1 9/10 0.14
0081 4 22 25 1 4/4 0.20
0162 7 25 35 8 18/20 0.15
0163 8 16 23 7 14/17 0.20
Most conflicts could be solved by introducing c-label Only few conflicts could not been solved Problematic regions are even hard to been labelled by experts Normally few hand-labels are necessary
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1616
Results on automatic segmentations
Method of Soares et al. and test database DRIVE of Staal High demands on segmentation algorithm:
Different vessel width, no gaps in segmentation, low false positive rate, etc. Some segmentations leads to poorly connected graphs (less rules)
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de
1717
Summary and Conclusions
We have developed a method for propagating vessel classification Requirement is a binary vessel image Problem is formulated as Constraint Search Problem Arising conflicts are solved by manipulating graph structure Interactive environment is developed for physicians
Methods works good for tested image databases Quality depends strongly on segmentation result
Further works– Statistical foundation of plausibility function– Realise initial labelling with Bayesian classifier– Justify method by comparison with ground-truth data– Enhance conflict solver– Classify strong vessel automatically as artery or vein – Integrate method in a framework for vascular structure analysis
Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins
GbR ’07GbR ’07
Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de