Image Management Dr. Hayit Greenspan Dept of BioMedical Engineering Faculty of Engineering [email protected] 640-7398.

Image Management

Dr. Hayit Greenspan

Dept of BioMedical Engineering

Faculty of Engineering

[email protected]

640-7398

Roles for Imaging in Health Care:

Diagnosis

Assessment and Planning

Guidance of Procedures

Communication

Education and Training

Research

Image Diagnosis in Dermatology

Fetus Ultrasound

Example of cross-sections through several parts of the body: skull, thorax, and abdomen,

obtained by computed tomography.

Visualization of the values of the attenuation coefficients by way of gray values produces an anatomic image.

Spinal cord Brain section

MRI Image Diagnosis

Roles for Imaging in Health Care:

Diagnosis

Assessment and Planning

Guidance of Procedures

Communication

Education and Training

Research

A functional map (in color) in the cerebellum during performance of a cognitive peg-board puzzle task, overlaid on a T2*-weighted axial image in gray scale. The dentate nuclei appear as dark crescent shapes at the middle of the cerebellum due to iron deposits. fMRI images were acquired by conventional T2*-weighted FLASH techniques with a spatial resolution of 1.25x1.25x8 mm3 and a temporal resolution of 8 seconds. Each color represents a 1% increment, starting at 1%. R, right cerebellum; L, left cerebellum. A left-handed subject used the left hand to perform the task. Bilateral activation in the dentate nuclei and cerebellar cortex was observed. The activated area in the dentate nuclei during performance of pegboard puzzle was 3-4 times greater than that seen during the visually guided peg movements. (see details in Kim et al., 1994b).

fMRI

Whole brain functional imaging study during a visuo-motor error detection and correction task. Functional images were acquired by the multi-slice single-shot EPI imaging technique with spatial resolution of 3.1x3.1x5 and temporal resolution of 3.5 seconds. The skull and associated muscles were eliminated by image segmentation. The 3-D image constructed from multi-slice images was rendered by Voxel View program (Vital Images, Fairfield, Iowa).The task was to move a cursor from the central start box onto a square target by moving a joystick. Eight targets were arranged circumferentially at 450 angles and displaced radially at 200 around a central start box. Activation (in color) is observed at various brain areas. Top image displays the brain as a 3-D solid object so that only the cortical surface is seen. In the bottom image, a posterior section was removed at the level of the associative visual cortex to display activation not visible from the surface (Kindly provided by Jutta Ellermann, Jeol Seagal, and Timothy Ebner).

fMRI

Medical Image Databases

• Medical Images are at the heart of diagnosis, therapy and follow-up.

• Digital medical image data in US per year:

bytes (petabytes).

• Generation & Acquisition

Post processing & Management.

• Medical imaging information types:

still images; pictures; moving images; structured text; plain text; sound; graphics.

• Driving the shift toward multimedia applications in medical imaging:

market demand; capital investment in imaging devices; need to organize and store multimodal image data + associated clinical data; ability to extract info in images.

1510

Biomedical Imaging

Structural Functional

MRI

X-ray CT Microscopy

Ultrasound

Projectionalx-ray

CR

DSAMammograph

fMRI MedicalopticalimagingEmission

CT

PET SPECT

Modality Image

Dimension(pixels)

Gray Level(bits)

Avg. Size(Mbytes)

MRI 256x256 12 8-20

Ultrasound 512x512 8 5-10

DSA(per run)

512x512or1024x1024

8 100-500

Current Information Systems

Originators

Publishers

Libraries

Users

Value-addedIndex Services

Digital Libraries

Originators

Users

Service X

Repositories

Multimedia Information Systems:Work-centered Scenario

Databases

Co-workers/Collaborators

Legacy DocumentsPhotos

Maps

OtherCollections

Visual Information Systems

Example:

Patient needs neurosurgery to remove a tumor– CT, MRI, PET scans: digitized and scanned

– Images are registered with a 3D brain model

– Locate tumor

– Path planning

– Using tumor as template, request to find:• patients of same sex• with similar tumors• in similar positions

Imaging Informatics

• Information systems and networks that facilitate the

Acquisition

Storage

Transmission

Processing

Analysis

Management

of medical images.

• Imaging Informatics- a new discipline:

Image generation

Image management

Image manipulation

Image integration

Basic concepts in Image Manipulation

• Global Processing: enhance contrast resolution;

• Segmentation: finding regions of interest;

• Feature detection & extraction;

• Classification;

Examples:

• Histogram equalization

• Temporal subtraction (DSA)

• Screening

• Quantitation

• 3D reconstruction and visualization

• Multimodality image fusion

Contrast enhancement

Principle of contrast enhancement: (a) intensity distribution along a line of an image;(b) same distribution after injection of the contrast medium; (c) intensity distributionafter subtraction; (d) intensity distribution after contrast enhancement.

Example of digital subtraction angiography (DSA) of the bifurcation of the aorta

An initial image mask is obtained digitized and storedContrast medium is injectedNumber of images are obtained.Mask is subtractedThe resulting image contains only the relevant informationThe differences can be amplified so the eye will be able to perceive the the blood vessels.Quality of deteriorate due to movements of the body can be corrected to some extent.

Texture Segmentation of MRI images

VOXEL-MAN(Hamburg): 3D Visualization

Atlasas of brain and other organs: allow views from any viewpoint;

Fusion of modalities +Anatomical atlases

http://www.uke.uni-hamburg.de/institute/imdm/idv/index.en.html

Video: COVIRAComputer Vision in Radiology

Basic concepts in Image Management

• Digital acquisition of images offers the exciting prospect of reducing the physical space requirements, material cost, and manual labor of traditional film-handling tasks, through online digital archiving, rapid retrieval of images via querying of image databases, and high-speed transmission over communication networks.

• Researchers are working to develop such systems that have such capabilities - picture archiving and communication systems (PACS).

• Issues that need to be addressed for PACS to be practical:– technology for high-resolution acquisition– high capacity storage– high-speed networking– standardization of image-transmission and storage formats– storage management schemes for enormous volumes of data– design of display consoles/workstations

Evolution of Image Management in PACS

• Early attempts in mid 80s– Univ. of Kansas, Templeton et al (84): earliest prototype systems to study PACS in

radiology– Inst of radiology in St. Louis, Blaine et al (83): PACS Workbench

experiments in image acquisition, transmission, archiving and viewing

• Substantial progress on several fronts:– Standards (DICOM) support transition from acquisition devices to storage devices– Expansion in disk capacities and dramatic decreases in cost– Hierarchical storage-management schemes – Compression methods– Increased resolution workstation display– Image manipulation tools

• Many Departments have mini-PACS; Large scale PACS increased in number from 13 to 23 in a 15-month period.

Image Management:

Indexing & Retrieval

We formed image archives

How do we access the content??

Extract content from file headers

Add Keywords

***Content-based Image Retrieval***

Visual Information Systems

Storage

IndexingSearch

& Retrieval

Representation

Retrieval

Visual Information

Representation

Indexing

Search &

Retrieval

Global HistogramsLocal Regions

Trees...

Search for:“Example like this”“similar image features”“50% blue and 50% green”

Color, textureshape...

Feature types

Which features should we use?How are we to organize them?Prioritize?Arrange for Search?

Examples of search queries

Visual Representation

• Text/Keywords wont do it:

“ One picture is worth a thousand words”

• Standard Object Recognition wont do it

• Our Representation & Indexing Goals– retrieve visual data based on content

– domain independent

– automated

Image Representation

• Image Processing

• Computer Vision

• Image Representation: Pixels to Content

Multimedia Object

Insertion

Feature Processing Module

Stored Features Query Features

Calculate Similarity

QueryMultimediaObject

Image Similarity

Storage and Retrieval of Images and Video

User InterfaceUser Interface

Content-Based Retrieval

Content-Based Retrieval

OrganizationOrganization

Database Management

Database Management

MetadataMetadata DatabaseDatabase

Content-based Information Retrieval

Scene Change Detection

Key-FrameExtraction

Image Pre-Processing

Camera &Object Motion

Camera Motion

Object Motion

Object

ShapeSketch

TextureColor

Spatial Relationships

Feature Extraction & Representation

Organization Module:

• Efficient query processing necessitates organization of indices for efficient search

• Image/Video indices:– are approximate– interrelated multiple attributes– not ordered

• Need flexible data structures (quad-tree, R-tree..)

Database Management Module

Physical storage structure and access path to the database• insulation between programs and data• provides a representation of the data• supprots multiple views of data• ensures data consistency

Video: Image GuidedDecision Support System for Pathology, Univ. of Rutgers

Evaluation Criteria for Image Retrieval Systems:

Automation

Multimedia Features

Adaptability

Abstraction

Generality

Content Collection

Categorization

Compressed Domain

Networked Multimedia for Medical ImagingRadiology Informatics Lab,

Univ. of San Francisco

Medical Image DBMS

Data sources

Post-processing

Communication

Visualization

Multimedia application 2

Multimedia application N

Multimedia application 1

Networked Multimedia for Medical ImagingRadiology Informatics Lab,

Univ. of San Francisco

Multimedia Medical Imaging Applications testbed:

• Bone age assessment• Temporal lung node analysis• Collaborative image consultation• Noninvasive neurosurgical planning