Institute for vision and graphics university of siegen, germany High-Level User Interfaces for Transfer Function Design with Semantics Christof Rezk Salama.

institute for vision and graphics university of siegen, germany

High-Level User Interfacesfor Transfer Function Design with Semantics

High-Level User Interfacesfor Transfer Function Design with SemanticsChristof Rezk Salama (Univ. Siegen , Germany)

Maik Keller (Univ. Siegen, Germany)

Peter Kohlmann (TU Vienna, Austria)

christof rezk-salama, institute for vision and graphics, university of siegen

Volume VisualizationVolume Visualization

Volume visualization techniques are mature from the technical point of view.

Real-time volume graphics on commodity PC hardwareMultidimensional transfer functions/classificationGradient estimation and local illumination on-the-flyMemory management and compression for large volumesEven global illumination techniques.

Is the ”volume rendering problem“ solved?

If you ask the computer scientist, he‘ll probably say „yes“.If you ask the users, they will most likely say „no“

christof rezk-salama, institute for vision and graphics, university of siegen

QuestionsQuestions

Why are volume rendering applications so hard to use for non-experts? Are volume rendering applications easy to use for us, the „experts“ ?What features must appropriate user interfaces provide?

christof rezk-salama, institute for vision and graphics, university of siegen

The Mental ModelThe Mental ModelExample taken from: Donald A. Norman

The Psychology of Everyday Things

christof rezk-salama, institute for vision and graphics, university of siegen

Volume VisualizationVolume Visualization

Transfer Function Design: Mapping of scalar data to optical properties (emission/absorption)Color table: Example: 1D TF for 12 bit Data, 4096 values x RGBA = 16384 DOF

Editors based on geometric primitives

1D Transfer Functions 2D Transfer Functions

christof rezk-salama, institute for vision and graphics, university of siegen

User IntentionUser IntentionExamples:

„Fade out the soft tissue“„Sharpen the blood vessels“„Enhance the contrast“

Question: What actions are necessary?Even the expert, who programmed the user interface, doesnot know this!

Mental model is inappropriate or missing!Semantics are missing (leads to “gulf of execution”)Result in trial-and-error

christof rezk-salama, institute for vision and graphics, university of siegen

Application

Abstraction LevelsAbstraction Levels

Low-Level Parameters(Color Table)

High-Level Parameters(Primitive Shapes)

Semantic LevelVisibilitySharpnessContrast

User

All previous approaches aim at reducing the complexity, the degrees of freedom.

None of the prevous approachestries to provide an appropriate mental model!

christof rezk-salama, institute for vision and graphics, university of siegen

Semantic ModelsSemantic ModelsRestrict ourselves to one specific application scenario.Example: CT angiography from neuroradiology

The visualization task will be performed manually for multiple data sets.Visualization expert and medical doctor!

Evaluate statistical information about the results:

Which parameter modifications are necessary to „make the blood vessels sharper?“

Use dimensionality reduction (PCA) to create a semantic model

christof rezk-salama, institute for vision and graphics, university of siegen

Bone

Step 1: Create a template for the TF

Brain/Soft Tissue Skin/Cavities Blood vessels

Developing a Semantic ModelDeveloping a Semantic Model

christof rezk-salama, institute for vision and graphics, university of siegen

Step 2: Adapt the template to reference data

Developing a Semantic ModelDeveloping a Semantic Model

christof rezk-salama, institute for vision and graphics, university of siegen

Step 2: Adapt the template to reference data

Developing a Semantic ModelDeveloping a Semantic Model

christof rezk-salama, institute for vision and graphics, university of siegen

Step 2: Adapt the template to reference data

Developing a Semantic ModelDeveloping a Semantic Model

Step 3: Dimensionality reduction

ReferenceTransfer Functions

Principal Component Analysis

Semantics

Semantic Model

christof rezk-salama, institute for vision and graphics, university of siegen

High-Level User Interface

High-Level Control

Transfer Function Semantic Model

Semantic ModelSemantic Model

christof rezk-salama, institute for vision and graphics, university of siegen

Semantic ModelSemantic Model

christof rezk-salama, institute for vision and graphics, university of siegen

Prototype ImplementationPrototype Implementation

Applicable to „anything that can be described by a parameter vector“

Take care of the scale!PCA for entire parameter vector is not appropriateSmall details might be missed

Our solution:• Split transfer function into

entities (=structures, groups of primitives with same scale)

• Perform PCA separately for each entity

• Reassemble the transfer function from the different entities

christof rezk-salama, institute for vision and graphics, university of siegen

ResultsResultsCTA: intracranial aneurysms:

512 x 512 x {120-160} @12bit, 100ml non-ionic contrast dye20 data sets for training / 5 data sets for evaluation

MR brain surgery:256 x 256 x {150-200} @12bit (noisy, lower dynamic range ~10bit)10 data sets

Evaluation of the model:Analytically: Stability of the eigenvectors (dot

product > 0.9)Stable for >12 data sets (regardless of individual choice)

User Study: Labels removed from the user interfaceMost semantics were correctly identified by non-expert users

christof rezk-salama, institute for vision and graphics, university of siegen

ConclusionConclusionUser Interface Design Strategies:

Reducing DOF is not enough.Good user interfaces must provide an appropriate mental model

Not an attempt to create a single user interfaces for any visualization tasksCreate semantic models for examination tasks as specific as necessaryBuilding block for software assistants for medical diagnosis and therapy planning

christof rezk-salama, institute for vision and graphics, university of siegen

AcknowledgementsAcknowledgements

Bernd Tomandl MD, Neuroradiologie, BremenChristopher Nimsky MD, Neurochirurgie, Erlangen