TECHNICAL REPORT … · Ivan Viola∗ Miquel Feixas† Mateu Sbert † Meister Eduard Gro¨ller∗ ∗Institute of Computer Graphics and Algorithms, Vienna University of Technology,

Institut fur Computergraphik undAlgorithmen

Technische Universitat Wien

Karlsplatz 13/186/2A-1040 Wien

AUSTRIA

Tel: +43 (1) 58801-18601Fax: +43 (1) 58801-18698

Institute of Computer Graphics andAlgorithms

Vienna University of Technology

email:[email protected]

other services:http://www.cg.tuwien.ac.at/ftp://ftp.cg.tuwien.ac.at/

TECHNICAL REPORT

Importance-Driven Focus of Attention

Ivan Viola Miquel Feixas Mateu Sbert† Meister Eduard Groller∗

TR-186-2-06-02

April 2006

Keywords: illustrative visualization, volume visualization, interacting with volumetric datasets, optimalviewpoint estimation, focus+context techniques

Importance-Driven Focus of Attention

Ivan Viola∗ Miquel Feixas† Mateu Sbert† Meister Eduard Groller∗

∗Institute of Computer Graphics and Algorithms, Vienna University of Technology, Austria†Institute of Informatics and Applications, University of Girona, Spain

Figure 1: Focus of attention applied to visual inspection oforgans within the human torso.

Abstract

This paper introduces a concept for automatic focusingon features within a volumetric data set. The user selectsa focus, i.e., object of interest, from a set of pre-definedfeatures. Our system automatically determines the mostexpressive view on this feature. An optimal viewpointis estimated by a novel information-theoretic frameworkwhich is based on mutual information measure. View-points change smoothly by switching the focus from onefeature to another one. This mechanism is controlled bychanges in the importance distribution among featuresin the volume. The highest importance is assigned tothe feature in focus. Apart from viewpoint selection, thefocusing mechanism also steers visual emphasis by as-signing a visually more prominent representation. To al-low a clear view on features that are normally occludedby other parts of the volume, the focusing also incorpo-rates cut-away views.

CR Categories: I.3.3 [Computer Graphics]:Picture/Image Generation—Display algorithms; I.3.3[Computer Graphics]: Picture/Image Generation—Viewing algorithms;

Keywords: illustrative visualization, volume visualiza-tion, interacting with volumetric datasets, optimal view-point estimation, focus+context techniques

∗{viola | meister}@cg.tuwien.ac.at†{feixas| mateu}@ima.udg.es

1 Introduction

Visualization is an application-oriented research areacombining knowledge from various fields of scienceor daily life and representing it by means of graph-ical elements. There can be many reasons for visu-alization [15]: visual analysis and visual presentationof underlying data are the most important ones. Thefirstly mentioned reason for visualization helps scien-tists to find relations and correspondence in various nat-ural phenomena. The visual stimulus is the strongestperception cue and visual analysis helps people tothinkvisually. The second goal serves as a communicationmedium and can be motivated by several reasons such aseducation, infographics, or commercial purposes. Theapproach of this paper helps in improving visual pre-sentation rather than visual analysis.

Recent developments of 3D scanning modalities such ascomputed tomography (CT) allows to unveil insights ofdifferent species, material, or bodies. One of the mostimportant application areas is medical diagnostic imag-ing. Besides medical visualization there are other sci-ence directions exploiting 3D scanning technology. Forexample high-resolution CT scanning has been appliedto provide Visible Man datasets or recently the mummydata set of Egypt’s boy pharaoh Tut. Industrial scannersare used for material quality validation, but can be usedfor scanning of small species, e.g., insects as well [24].An interesting example of exploiting 3D scanning tech-nology is theDigital Morphology library [5], a large dy-

1

2 2 RELATED WORK

namic archive of information on digital morphology ofbiological specimens.

As medical imaging is an important application area,medical workstations in general feature the broadestspectrum of functionality for handling volumetric datasets. This includes visualization, image processing,measurements, or (semi-)automatic diagnosis estima-tion. Medical workstations, however, are designedmostly for visual analysis in diagnostic scenarios, ratherthan for presentation purposes. Medical workstationscurrently do not include much functionality that is nec-essary for presentation purposes.

Other areas of science on the other hand are in gen-eral not targeted specifically to visual analysis and vi-sual presentation is often more required. One of theseexamples is the collection of different specimens in theDigital Morphology library. The aspect of visual pre-sentation is also becoming important in communicationamong medical experts from different domains or be-tween the medical staff and the patient. Therefore func-tionality for presentation purposes, also strongly relatedto visual storytelling, will become more important formedical workstations as well. Functionality that is han-dling volumetric data sets for presentation purposes hasbeen recently discussed in a tutorial onillustrative visu-alization [21]. Some systems incorporating illustrativepresentation techniques are shortly discussed in the re-lated work (Section 2).

Current visualization systems for handling volumetricdata sets require a lot of expertise from the user. For ex-ample many widgets to design a suitable transfer func-tion (mapping tissue density to color and opacity val-ues) are rather unintuitive for the unexperienced user.Our work is motivated by the fact that currently noneof the commercially or publicly available visualizationsystems allows the user high-level interactions such as”Show me this interesting part of the volumetric data setand then show me the next interesting part.” Our frame-work allows an automatic focus of attention on interest-ing objects. The user’s only required (but not limitedto) interaction is to select an object of interest from aset of pre-segmented objects. The framework smoothlynavigates the view to optimally see the characteristics ofthe focus object. Additionally, the focus object is visu-ally emphasized for easy discrimination from the con-text. Example images that illustrate focus of attentionfor insights of a human torso and human hand datasetare shown in Figures 1 and 2.

One contribution of this paper is the introduction of aninformation theoretic framework for optimal viewpointestimation in volumetric data sets with pre-segmented

objects. This framework easily integrates the impor-tance of objects within the volumetric data set. Anothercontribution is a concept of focus of attention for inter-active volume visualization. Here an expressive view-point is selected in combination with a visually pleasingdiscrimination of focus from context information. Bychanging the object of interest, both viewpoint settingsand visual parameters are smoothly changing to put em-phasis on the newly selected object of interest.

The paper is organized as follows: Section 2 describesprevious work related to importance-driven focus of at-tention. The following Section 3 describes the conceptof focusing. Technical details of optimal viewpoint es-timation are discussed in Section 4. Interaction aspectsof focusing are presented in Section 5. Implementationissues and performance are discussed in Section 6. Fi-nally we draw conclusions and summarize the paper inSection 7.

2 Related Work

Focus of attention has been often used in visualization tocatch the user’s attention. It has many different occur-rences. We will first review relevant previous work inthe area of focus+context visualization. The second partof this section reviews recent work on optimal viewpointestimation as good viewpoint selection is crucial for aneffective focus of attention.

The depth of field effect is a focus of attention tech-nique from photography that inspired Kosara et al. [8]to propose a semantic depth of field (SDOF). In theirwork they have shown that the degree of sharpness de-termines the speed of drawing human attention in oth-erwise blurry environments. They have applied theirtechnique in various fields of information visualization.Later on, the authors have designed a user study fora quantitative evaluation of sematic depth of field ef-ficiency [9]. They show that the semantic depth offield is an effective way to draw attention to specificparts. SDOF, however, should be used in combinationwith other visualization techniques. Additionally theusers sometimes felt uncomfortable observing unnatu-rally blurred parts (e.g., blurred text).

Another focus+context method for displaying volu-metric data has been proposed in previous work onimportance-driven volume visualization [22, 23]. Theimportance classification has been introduced for spec-ifying view-dependent visual representations to revealoccluded structures. This is in spirit of cut-away viewsand ghosted views known from traditional illustrations.

3

Figure 2: Further examples of visual inspection of or-gans within the human torso.

Several focus+context techniques have been includedinto VolumeShop, a publicly available volume visual-ization system from our group [4]. Besides applyingcut-aways and ghosted views, the user can manipulatefeatures of interest in several ways, e.g., displace themfrom their original spatial location to a part of imagespace, where otherwise no data is shown. Addition-ally to focus+context techniques, the user can enrich thevisualization by adding textual information to objectswhich appears as automatically placed labels. The func-tionality of VolumeShop is intended to provide a toolfor presenting and communicating the data being visu-alized.

Another publicly available visualization system featur-ing functionality for visual presentation and communi-cation has been proposed by Svahkine et al. [13]. Aninteresting consideration incorporated in their system isthe level of expertise of the user. This has two implica-tions for the design of the system. First, the user inter-face and widgets are customized according to the user-expertise level. A non-expert user has a very simple userinterface allowing limited flexibility, whereas an experthas much higher flexibility with advanced tools such asa transfer function editor. Second, the level of user ex-pertise implies also different visualization results. Aneasy to understand visualization is targeted to a non-

expert user and moredirect visualization is targeted tothe expert.Viewpoint selection has been applied to several do-mains in computer graphics, such as scene understand-ing and virtual exploration [1], molecular visualiza-tion [20], image-based modeling [19], volume visual-ization [3, 14], and mesh saliency [10]. Different mea-sures for viewpoint evaluation have been used in thesefields.Vazquez et al. [18] have defined theviewpoint entropy(Equation 13), as a measure for viewpoint quality eval-uation. This measure has been designed primarily forpolygonal data, where the best viewpoint is defined asthe one that has maximum entropy. Taking into accountthe background information, this technique may be usedfor indoor and outdoor scenes as well.Viewpoint entropy for polygonal data has been recentlyextended to volumetric scalar data [3], by substitutingthe area visibility distribution by the voxel visibility dis-tribution divided by the voxel importance (noteworthi-ness factor). This work has also suggested information-theoretic measures for clustering views according tosimilarity using the Jensen-Shannon divergence from in-formation theory (Equation 12). They also suggested anoptimal viewpoint estimation scheme for time-varyingdata.It has been shown recently by Sbert et al. [12] thatviewpoint entropy is very sensitive to triangulation.The maximum entropy is achieved in areas of veryfine triangulation. Therefore they propose a newviewpoint-quality measure for polygonal data based onthe Kullback-Leibler distance (KL) (Equation 11) de-noted as viewpoint KL distance (Equation 14). Theviewpoint KL distance is interpreted as the distance be-tween the normalized distribution of projected areas andthe ideal projection, given by the normalized distribu-tion of the actual areas. In this case, the backgroundis not taken into account. The minimum value 0 is ob-tained when the normalized distribution of the projectedareas is equal to the normalized distribution of the actualareas. Thus, views of high quality correspond to viewswith minimal KL distance. One drawback of this mea-sure is that many non-visible or poorly visible polygonsin a model can distort the quality of the measure.

3 Focusing Considerations

Before going into technical details of our work wewould like tofocus the reader’sattention on several con-siderations we have made during designing our frame-work. To get a clear high-level overview on the frame-work functionality, we briefly present the processing

4 3 FOCUSING CONSIDERATIONS

pipeline. Technical details will follow in Sections 4and 5.

Focus discrimination: Focus of attention is a visualdiscrimination of interesting objects from other ele-ments in an image. It is realized through visual empha-sis of the object of interest while other objects presentedas context are suppressed. In general a discrimination ofthe focus from the context can be achieved by differentlevels of sparseness in their visual representation [22].The focus is represented verydensely while the con-text gets a moresparse visual representation. Levels ofsparseness can be designed in many ways. In photog-raphy for example, a very effective technique for objectdiscrimination is the sharpness of the object of interest.Very sharp objects are automatically perceived as beingin focus, more blurry objects are contextual information.Levels of sparseness are in this case different sharpnesslevels. Recent studies [9] have shown that for visualiza-tion tasks the modulation of sharpness is not very muchpreferred by users. In volume visualization tasks thedepth-of-field from photography may additionally con-flict with visual artifacts such as partial volume effects.In this case opacity, color brightness, and saturation canbe used to discriminate the most interesting objects fromthe rest in a much clearer way.

Characteristic view: In addition to visual discrimina-tion, objects in focus have to be shown from a char-acteristic view where most of the focus structures areperceivable. The most interesting object must not beoccluded by less relevant parts. If possible the focusshould be in front of other features. In case that the fea-ture of interest is always occluded by other features, cut-away views or other concepts from illustration can be in-cluded into the visualization. In this case it is importantthat the cut-away region does not entirely remove otherinteresting objects. If possible, only the least relevantobjects are cut away. Furthermore a proper orientationof the up-vector of the viewpoint and a proper position-ing of the focus to fulfill aesthetical criteria of composi-tion (e.g., rule of thirds [6]) are important to consider inthe viewpoint specification. All mentioned aspects indi-cate that a proper viewpoint setting is important for thefocus of attention.

Focusing Pipeline: Some previous work [22, 23] usedan explicit importance classification for focus+contextvisualization inspired by techniques known from tradi-tional illustration. In the following we give an overviewon the pipeline of importance-driven focus of attention(for the part on optimal viewpoint estimation see alsoFigure 3). The pipeline is presented for the visualizationof volumetric datasets. The concept is, however, univer-sal and can be applied to various visualization tasks irre-

spective from the type of the underlying data. The def-inition of levels of sparseness for visual representationsis highly application dependent. They can be defined ex-plicitely by the user, estimated (semi-)automatically [7],or selected from design galleries [11]. A discussion onlevels of sparseness is outside the scope of this paper.We concentrate on the estimation of proper viewpointsand on aspects of focusing during user interaction.

Finding a viewpoint where the characteristics of a spe-cific feature are clearly visible is crucial for focus of at-tention. This naturally requires the estimation of visi-bility of the feature under specific viewing settings. Inour case, i.e., for objects within the volumetric data set,this process is rather time-consuming as it requires raycasting of the whole data set from various viewpoints.Computing the visibility of features on-the-fly duringinteraction will strongly limit interaction possibilities.The visibility of features depends on their visual repre-sentation. For applications where a frequent change ofvisual representations is not relevant, the visibility es-timation can be easily treated as a pre-processing step,which is executed once prior to the user interaction.

In our importance-driven optimal-viewpoint estimationframework we compute the visibility of an object as itscontribution on the finally rendered image. This com-putation is based on the opacity contribution of eachvoxel belonging to the object. Object visibility is thenmapped to aconditional probability of the object for agiven viewpoint. These values are used for computationof good viewpoints for a given object. We use for this anovel information-theoretic framework combined withobject importance information as described in detail inSection 4.

With selecting visual representations of tagged objectsand by identifying representative viewpoints, the cru-cial information to perform on-the-fly focus of atten-tion is available. We use focus of attention as a toolfor visually-pleasingbrowsing among a number of in-teresting structures within the data. Browsing can be ingeneral used for visual presentations of the volumetricdata. In our importance-driven focusing framework wealso consider additional information about the data. Forexample we include information about theup-vector ofthe volume (e.g., in the case of the human anatomy, thehead is on top and the legs are at the bottom), in order topreserve natural orientations of viewpoints. The objectin focus is located in the center of the viewpoint in orderto draw the maximal attention of the user. We blend-intextual information as labels to increase the semantic in-formation content. In general, the more information isavailable the larger the spectrum of possibilities how to

4.2 Information-Theoretic Framework for Viewpoint Estimation 5

realize a pleasing focus of attention. Browsing is real-ized as a continuous change of the focus of attention.Visual representations and viewpoint settings continu-ously change to visually emphasize the newly selectedobject of interest. These changes are driven by changesin the importance distribution among objects. A detaileddiscussion on browsing through the structures is given inSection 5.

4 Characteristic Viewpoint

In this section, we describe our approach for selecting acharacteristic viewpoint for a particular object. First, wedetermine the visibility of structures within the volumet-ric data considering their visual representations. Thenwe use the visibility as input to the new information-theoretic framework. This framework integrates per-object importance classification, which allows to esti-mate optimal viewpoints for an object within the vol-ume.

4.1 Visibility Estimation

The first step for a viewpoint evaluation is the estima-tion of per-object visibility. We use a simple scheme forvisibility evaluation, taking into account opacity contri-bution of voxels on the rendered image. The evalua-tion of the visibility is done in a ray-casting step. Foreach samplei along a rayr we evaluate its visibilityv(r, i) = v(r, i−1)α(r, i), whereα(r, i) is the resampledopacity value at the given sample positioni. We imple-ment nearest neighbor and linear interpolation resam-pling schemes. The visibility of a voxel is given as thesum of visibilities of all resampled points the voxel iscontributing to in the resampling step. In case of near-est neighbor interpolation we simply sum the ray samplevisibilities belonging to this voxel. In case of linear in-terpolation, we perform a lineardistribution of the raysample visibility among all eight surrounding voxels.

Each voxel belongs either to a particular feature or it be-longs to thebackground volume. The sum of voxel con-tributions belonging to a particular feature, estimates thevisibility of the feature. We are using non-binary objectclassifications and a particular voxel may contribute to anumber of different features simultaneously. The voxelvisibility is simply multiplied by a factor that defineshow much the voxel contributes to a particular object.

In our focus of attention framework, we also changethe visual representation of the object of interest. This

means that the visual representation is not constant dur-ing the time of interaction. This has to be taken into ac-count while computing visibilities. Therefore we com-pute the visibility for eachactive object, i.e., object infocus. This means, for each viewpoint we get(n + 1)different visibility values forn objects. Each object isset once as active object and once the visibility is com-puted with no selected active object. When we searchfor the optimal viewpoint of a particular object, we usethose visibilities where this object has been the activeobject. In this case the object has a different visual rep-resentation from the rest of the volume.

One problem that arises when computing the visibilityof objects, is that some features may be completely oc-cluded by other features. This is caused by very densevisual settings. This will mean that there is no opti-mal viewpoint from which the feature is clearly visible,or all viewpoints are equally good or bad. In order todeal with this problem, we have optionally included cut-away views in the visibility estimation. Here the activeobject is visible from all viewpoints as the volume re-gion in front of this object is not visible at all.

The above described visibility evaluation does not con-sider the location of features in image space. To drawattention to a feature, it is important that it is locatedclose to the center of the image. To bring the featureinto the focus, we give more prominence to rays in thecenter of the image. We weight each ray’s contributionto the visibility of objects and background volume by animage space weight. This weight is largest in the centerof the image and is decreasing with the distance fromthe center.

The overall concept of optimal viewpoint estimationdriven by an importance distribution is illustrated in Fig-ure 3. The importance distribution and the visibility ofeach object for the given visual representations are in-put parameters of the information-theoretic framework.This framework will be described in detail in the nextsection.

4.2 Information-Theoretic Framework forViewpoint Estimation

After the visibility of each object under different visualsettings and viewpoints has been computed, the optimalviewpoint estimation can be performed. In this sectionwe describe our approach for finding good viewpoints.Our viewpoint selection approach is using the mutual in-formation of the channel defined between a set of view-points and the objects of a volumetric data set. This newmeasure shows a better behavior and robustness than theprevious viewpoint entropy [18]. For more information

6 4 CHARACTERISTIC VIEWPOINT

o2 o3o1

importance distribution

o1

o2

o3

object selection by user v1

v2

v3

o1

o2

o3

visibility estimation

image-space weight

p(v1)

p(vn)

p(o1|v1)

p(om|vn)

p(o1) p(om)

...

...

...

I(vi,O) = p(oj|vi) logΣj

mp(oj|vi)p(oj)

...

...

...

information-theoretic framework

for optimal viewpoint estimation

Figure 3: Concept of importance-driven optimal view-point estimation.

on information-theoretic measures and previous view-point quality measures, please refer to the Appendixsection.

Our framework works well for volumetric objects astagged volume regions. Taking a voxel as a basic ob-ject element would lead to very high memory consump-tion, this will be also the case using previously sug-gested viewpoint quality measures. The frameworknaturally integrates per-object importance classification.By changing the importance distribution among objects,the results of viewpoint evaluation also change to have acharacteristic view on the feature of highest importance.Setting the importance to be constant for all objects andbackground volume, characteristic views for the entirevolume are achieved.

We formalize the viewpoint selection using a commu-nication channel between two random variables (inputand output). The communication channel is character-ized by a probability transition matrix which determinesthe output distribution given the input. After defining achannel, entropy and mutual information can be calcu-lated. The entropy is a measure of the average uncer-tainty in a random variable and the mutual informationis a measure of the dependence between two randomvariables, i.e., the amount of information one randomvariable contains about another. While entropy is theself-information of a random variable, mutual informa-tion is a special case of a more general quantity calledrelative entropy, which is a measure of the distance be-tween two probability distributions.

We first define a channelV → O between a set of view-points and the objects of a volumetric data set, repre-sented respectively by the random variablesV (input)andO (output). Viewpoints will be indexed byv andobjects byo. The marginal probability distribution of

V is given by p(v) = 1Nv

, whereNv is the number ofviewpoints, i.e., we assign the same probability to eachviewpoint. The conditional (or transition) probabilitiesp(o|v) are given by the normalized visibility of each ob-ject from each viewpoint, i.e.,∑o∈O p(o|v) = 1. Finally,the marginal probability distribution ofO is given by

p(o) = ∑v∈V

p(v)p(o|v) =1

Nv∑

v∈V

p(o|v), (1)

that expresses the average visibility of each object fromthe set of viewpoints.

From channelV → O, theconditional entropy is givenby

H(O|V )=− ∑v∈V

p(v) ∑o∈O

p(o|v) logp(o|v)=1

Nv∑

v∈V

Hv,

(2)whereHv = −∑o∈O p(o|v) logp(o|v) is the entropy ofviewpoint v (for polygonal data see Equation 13, forvolumetric data refer to Bordoloi et al. [3]). Thus, theconditional entropy is the average of all viewpoint en-tropies.

We now focus our attention on mutual information, thatexpresses the degree ofdependence or correlation be-tween a set of viewpoints and the data set. Themutualinformation betweenV andO is given by

I(V,O)= ∑v∈V

p(v) ∑o∈O

p(o|v) logp(o|v)p(o)

=1

Nv∑

v∈V

I(v,O),

(3)where

I(v,O) = ∑o∈O

p(o|v) logp(o|v)p(o)

(4)

is calledviewpoint mutual information and representsthe degree of correlation between the viewpointv andthe set of objects. The quality of a viewpoint is given bythe mutual informationI(v,O) and the best viewpointis defined as the one that has minimum mutual infor-mation. High values of the measure mean a high de-pendence between viewpointv and the set of objects,indicating a highlycoupled view. On the other hand,low values correspond to a low dependence, allowingfor morerepresentative views of the data set.Viewpoint mutual information has the following advan-tages versus viewpoint entropy. First, the entropy valueincreases to infinity with the number of voxels and itis highly dependent on the voxel distribution. Thus, anextremely refined mesh attracts the attention of the mea-sure, penalizing big objects in front of small ones. Thisis not such a problem for volumetric data sets storedon a regular grid, when the basic object element is avoxel. Viewpoint selection evaluation for volumetric

4.4 Obtaining Characteristic Viewpoints 7

data stored on unstructured grid will suffer from thisproperty much more significantly. On the other handthe viewpoint mutual information converges to a finitevalue when the mesh is infinitely refined and is insensi-tive to changes in the voxel resolution.

Second, while viewpoint entropy only uses the condi-tional distributionp(o|v), that is, what is visible from agiven point of view, viewpoint mutual information mea-sures how much the distributionp(o|v) differs from thedistributionp(o) in the sense of statistical distinctness.Note thatp(o) gives us the average visibility of all ob-jects captured from all viewpoints and represents theideal target, so that the viewpoint mutual information iszero whenp(o|v) = p(o). In other words, the viewpointmutual information considers all the information of thechannel. The only objective of viewpoint-entropy max-imization is to approach the uniform distribution, with-out taking into account the degree of visibility of theobjects. This behavior of the entropy is independent ofweighting the visibility distribution by the importance,as done by Bordoloi and Shen [3].

4.3 Incorporating Importance

We observe that viewpoint mutual information can berewritten as

I(v,O) = KL(p(O|v)|p(O)), (5)

where capital letters indicate thatp(O|v) is the condi-tional probability distribution betweenv and the data set,and p(O) is the marginal probability distribution ofO.Thus,I(v,O) can be interpreted as the relative entropyor Kullback-Leibler distance between the visibility dis-tribution of objects from viewpointv and their averagevisibility. The less the measure the better the viewpoint,as we approach the ideal target of viewing every objectproportional to the average visibilityp(o). In this case,I(v,O) would be zero.

Adding importance to our scheme means simply mod-ifying the target function. The ideal viewpoint wouldbe now the one viewing every object proportional to theaverage visibility times importance. After incorporatingimportance, the viewpoint mutual information is givenby

I′(v,O) = ∑o∈O

p(o|v) logp(o|v)p′(o)

, (6)

where

p′(o) =p(o)i(o)

∑o∈O p(o)i(o)(7)

andi(o) is the importance of objecto.

4.4 Obtaining Characteristic Viewpoints

Equation 6 defines the viewpoint mutual informationwith importance classification. This is computed foreach viewpoint and for each active object separately (asthey have different visual representations, which impliesdifferent visibilities). To obtain a set of characteristicviews for a given objecto, we compute the conditionalprobabilities of all objects for a given viewpoint. Theconditional probabilityp(o|v) is equal to the normalizedvisibility, i.e., the visibility of all objects per viewpointare equal to 1 as described in Section 4.2.

Furthermore we have to compute the marginal proba-bility p(o) from Equation 1. To computep′(o) we firstcompute a dot product between the marginal probabil-ity vector (p(o0), p(o1), p(o2), ..., p(om−1), p(om))and the importance distribution vector(i(o0), i(o1), i(o2), ..., i(om−1), i(om)) where m isthe number of objects ando0 is the background volume.After the sum in the denominator of Equation 7 is com-puted, all information is available and we can computethe viewpoint mutual information for viewpointv.

The viewpoint mutual information is computed for ev-ery viewpoint and the set of viewpoints with the smallestmutual informations are selected. These computationsgive us good viewpoints for a particular active object. Tocompute good viewpoints for another object, we have totake another set of visibilities where the visual emphasisis on the respective object. All values necessary for theviewpoint mutual information can be stored in a set of2D schemes as shown in Figure 3.

5 Importance-Driven Focusing

The focus of attention requires to display the object ofinterest from a characteristic view. How to obtain char-acteristic viewpoints has been described in the previoussection. Let’s assume we have identified a set of mostcharacteristic viewpoints per object under the given vi-sual representations. Now we will describe in detail howthe focus of attention can be used for the visual inspec-tion of tagged volumetric data.

The general idea is to use the importance distribution asa controlling parameter for the focus of attention. Wespecify a high importance value for the active object(e.g., 100.0). The other objects and the background areassigned a low importance value (e.g., 1.0). By selectinganother object to become the active object, the impor-tance of the previously selected active object is contin-uously decreasing to the value of inactive objects (1.0)and the importance of the newly selected active object

8 5 IMPORTANCE-DRIVEN FOCUSING

is increasing to the maximal value (100.0). The changein the importance distribution is reflected on the view-point location and visual representations. This causesa smooth and visually pleasing change of focus of at-tention to the newly selected active object. We describethese changes in more detail for viewpoints and visualrepresentations separately.

5.1 Viewpoint Transformation

Initially we set the viewpoint to optimally see the entirevolume without selecting any active object. All impor-tance values are set to a constant low importance value.For each object we calculate several good characteristicviewpoints, as well as several good viewpoints on theentire volume with no active object selected. All theseviewpoints are located on a bounding sphere around thevolumetric data set. After selecting an active object, itsimportance raises and the viewpoint changes to the opti-mal viewpoint of the active object. As there are severalcharacteristic per-object viewpoints, we have to definewhich one will be selected. To minimize the viewpointpath, we compute the angle between the current view-point’s normal vector and the normal vectors of the ob-ject’s good viewpoints. The viewpoint with the smallestangle is selected. The position of the viewpoint is al-ways on the bounding sphere. The change of positionbetween viewpoints is calculated as a linear transforma-tion from one position to another. Every position of anintermediate viewpoint is then normalized to be locatedon the bounding sphere surrounding the volume. Thishas one favorable implication. The viewpoint changestarts slowly, has the biggest angle difference in the mid-dle of the viewpoint transformation and slows down be-fore achieving the new optimal viewpoint. This is de-picted in Figure 4. The change of viewpoints is param-eterized by the importance value of the object in focus.The initial value for this object is low and equal to theimportance value of the other objects. After selectingthe object as active, the importance increases and de-fines the position of intermediate viewpoints. When anobject’s importance value is equal to the maximal value,the viewpoint location is at the desired position.

After achieving the object’s characteristic viewpoint, theuser can locally change the viewpoint in order to inspectthe interesting object from several directions. When thisinspection is finished (e.g., mouse button is released),the importance value of the active object is set to a lowvalue again. Increasing the importance brings the view-point back to one of the characteristic views.

We use a slightly different concept for changing view-points from one active object to another. Instead of a

v1 v2

o1 o2

o3

Figure 4: The viewpoint path is calculated as a differ-ence between two viewpoint positions. The path is thennormalized onto the bounding sphere, which smooth ac-celeration and deacceleration in viewpoint change.

direct viewpoint interpolation from the actual viewpointto theclosest new active object’s good viewpoint (deter-mined by the angle difference of viewpoint normal vec-tors), we consider that the viewpoint path visits a view-point that gives a general overview on the entire volume.This provides the context information of all structures sothe user does not lose its orientation within the volume.From thiscontext view, the viewpoint smoothly changesto the optimal viewpoint of the newly selected active ob-ject. This way of presenting objects is in the spirit of thenavigation on large 2D maps proposed by van Wijk andNuij [16, 17].In case of switching the viewpoint from one active ob-ject to another, the viewpoint change considers threepre-selected viewpoints: the optimal viewpoint of theprevious active object, the contextual overview, andthe view on the new active object. Instead of the lin-ear transformation discussed above, the position of theviewpoint is changing on a Bezier curve defined byviewpoint positions as three control points [2]. Thismeans that the contextual view on the whole volume isnot visited exactly, it is approximated by similar viewsthat also satisfy the goal of providing context. The view-point position is again normalized to the unit sphereenclosing the volume. The Bezier curve among threeviewpoints is depicted in Figure 5.In this case we have to select from the two closest op-timal viewpoints, i.e., one for the context view and onefor the characteristic view on the new active object. Weselect the viewpoint pair with the smallest sum of an-gles between the viewpoint normals, which means thatthe overall path is shortest.An important consideration in the viewpoint setting withrespect to a visually pleasing focusing, is the orienta-tion of the viewpointup-vector. In our implementationwe set the viewpoint up-vector to point towards the up-vector of the volume. The up-vector of the volume is

9

o1 o2

o3

vc

v1 v2

Figure 5: Change between two optimal viewpoint of dif-ferent objects (v1 for o1 andv2 for o2). The contextualviewpoint vc is nearly visited and is approximated bythe Bezier curve.

defined before the visual inspection for each dataset, to-gether with all other preprocessing steps: defining ob-jects by segmentation, visual representations and goodviewpoint estimation. If the viewpoint is located at thepoles, i.e., the viewpoint normal vector is parallel to thevolume top vector, we select another vector to be theviewpoint up-vector. In our implementation we use thevolume front vector with inverse orientation so we lookat the volume from top-front.

5.2 Visual Representation

A characteristic view is an important part of focus of at-tention. However without emphasis through visual rep-resentation, the focus is still not discriminated from thecontext objects. Therefore in our framework changesin importance distribution also change the visual ap-pearance of objects. A visual representation basicallychanges in a similar way as the viewpoint. In this casewe do not need to calculate a path. We select the appro-priate level of sparseness in the visual representation. Inour implementation we define the visual representationof inactive and active objects before the visibility calcu-lation. These visual representations can be linearly in-terpolated for example. In our focusing pipeline we usea discontinuous change in the visual representation asthis abrupt change attracts an observer’s attention muchstronger. While the viewpoint moves from the previ-ous active selection towards thecontext view (the im-portance of the previous active object is decreasing), theprevious active object is still visually emphasized. Afterreaching the context viewpoint, the visual representationof the previously active object is suppressed and the newactive object is visually emphasized (the importance ofthe newly selected active object is increasing).

In addition to changes in the visual representation, weincorporate cut-away views. The level ofghosting infront of the interesting feature is also driven by impor-tance changes. In this case we do not employ abruptchanges, but the level of ghosting changes smoothly.This means the ghosting level is increasing with de-creasing importance of the previously active object andis decreasing with increasing importance of the newactive object. When the optimal view is reached, theghosting level is maximal, i.e., features in front of theactive object are completely transparent. We include ad-ditional information into this static view by blending-inadditional annotations.

6 Results

We have integrated the focus of attention functionalityas a plugin into VolumeShop [4]. This system allowseasy prototyping with the possibility of using a lot ofexisting functionality. We have extended the informa-tion about the dataset, which is stored in an XML file,by information on the volume up-vector and on the vol-ume front-vector. After the viewpoint estimation, foreach object a set of characteristic viewpoints is savedinto the XML structure as well as theglobally charac-teristic views when no specific object is in focus. Vis-ibility computation for each object is the most time-consuming part of the pipeline and takes about few min-utes, because the ray-casting has to be performed for alarge number of viewpoints. As this is a pre-processingstep that is considerably shorter than object specifica-tion by segmentation or setting-up proper visual repre-sentations, this is not a real issue. Interaction is doneon-the-fly and additional viewpoint location computa-tions as well as importance-driven modifications of vi-sual representations do not take any noticeable time andthe performance is equal to standard multi-volume ren-dering implemented in VolumeShop [4].

Focus of attention has been applied on three differentdata sets. The human hand and torso (Figures 1 and 2)show objects that are inside the data set. In this casethe visibility computation used cut-away views to iden-tify the best visibility. In case of the stag beetle dataset (Figure 6), only outer parts have been selected so theoption for cut-away visibility calculation was not neces-sary. In this figure sample images have been taken fromre-focusing from thorax object to the legs. Between thefourth and fifth image the contextual viewpoint has beenreached and the focus switched to legs.

The concept of importance-driven focusingis best demonstrated by the accompanying

10 REFERENCES

Figure 6: Stag beetle data set: re-focusing from thorax object to the legs).

video. Further information is available athttp://www.cg.tuwien.ac.at/research/vis/exvisation/idf/.

7 Summary and Conclusions

In this paper we have proposed the concept forimportance-driven focus of attention. We have dis-cussed the necessary pre-processing steps before a vi-sual inspection puts the focus of attention on interest-ing objects. One of these steps is localization of view-points that show characteristics of an object in the bestway. We use a new method for viewpoint selection forvolume data using viewpoint mutual information thatworks very good for segmented volumetric data classi-fied by importance.

We have shown possibilities how to realize focus of at-tention for a visual inspection of volumetric data withadded information such as varying visual representa-tions, optimal viewpoints for objects and the entire vol-ume, up-vector of the volume, and auxillary textual in-formation.

We have discussed aspects of a visually pleasing re-focusing from one object of interest to another. Thisincludes the selection of viewpoints, design of a path forthe viewpoint and also changes in the visual representa-tion. Browsing through pre-selected structures gives agood overview on the information content of the under-lying data.

8 Acknowledgments

The work presented in this publication is car-ried out as part of the exvisation project(www.cg.tuwien.ac.at/research/vis/exvisation)supported by the Austrian Science Fund (FWF) undergrant no. P18322.

This project has been funded in part with grant numbersTIN2004-07451-C03-01, FIT-350101-2004-15 of theSpanish Government and IST-2-004363 (GameTools:Advanced Tools for Developing Highly Realistic Com-puter Games) from the VIth European Framework.

The stag beetle from Georg Glaeser, Vienna Universityof Applied Arts, Austria, was scanned with an industrialCT by Johannes Kastner, Wels College of Engineering,Austria, and Meister Eduard Groller, Vienna Universityof Technology, Austria. TheMonster Study human torsoand the human hand data sets are courtesy of Tiani Med-graph.The authors would like to thank Stefan Bruckner andPeter Rautek for fruitful discussions.

References

[1] C. Andujar, P. P. Vazquez, and M. Fairen. Way-finder: guided tours through complex walkthroughmodels. Proceedings of Eurographics ’04, pages499–508, 2004.

[2] P. Bezier. Emploi des machines a commandenum4rique. Masson et Cie., Paris, 1970. Trans-lated by R. Forrest and A. Pankhurst as Numeri-cal Control – Mathematical and Applications, JohnWiley and Sons Ltd., London, 1972.

[3] U. Bordoloi and H.-W. Shen. View selection forvolume rendering. InProceedings of IEEE Visual-ization’05, pages 487–494, 2005.

[4] S. Bruckner and M. E. Groller. Vol-umeShop: An interactive system for di-rect volume illustration. InProceedings ofIEEE Visualization’05, pages 671–678, 2005.http://www.cg.tuwien.ac.at/volumeshop/.

[5] Digital morphology web site,http://www.digimorph.org/, 2006.

[6] B. Gooch, E. Reinhard, C. Moulding, andP. Shirley. Artistic composition for image cre-ation. InProceedings of Eurographics Symposiumon Rendering ’01, pages 83–88, 2001.

[7] G. Kindlmann and J. Durkin. Semi-automatic gen-eration of transfer functions for direct volume ren-dering. InProceedings of IEEE VolVis ’98, pages79–86, 1998.

REFERENCES 11

[8] R. Kosara, S. Miksch, and H. Hauser. Semanticdepth of field. InProceedings of IEEE InfoVis ’01,pages 97–104, 2001.

[9] R. Kosara, S. Miksch, H. Hauser, J. Schrammel,V. Giller, and M. Tscheligi. Useful properties ofsemantic depth of field for better F+C visualiza-tion. In Proceedings of VisSym ’02, pages 205–210, 2002.

[10] C. H. Lee, Amitabh Varshney, and David W. Ja-cobs. Mesh saliency. InProceedings of ACM SIG-GRAPH ’05, pages 659–666, 2005.

[11] J. Marks, B. Andalman, P. A. Beardsley, W. Free-man, S. Gibson, J. Hodgins, T. Kang, B. Mir-tich, H. Pfister, W. Ruml, K. Ryall, J. Seims, andS. Shieber. Design Galleries: a general approach tosetting parameters for computer graphics and ani-mation. InProceedings of ACM SIGGRAPH ’97,pages 389–400, 1997.

[12] M. Sbert, D. Plemenos, M. Feixas, andF. Gonzalez. Viewpoint quality: Measures andapplications. InProceedings of ComputationalAesthetics ’05, pages 185–192, 2005.

[13] N. Svakhine, D. Ebert, and D. Stredney. Illustra-tion motifs for effective medical volume illustra-tion. IEEE Computer Graphics and Applications,25(3):31–39, 2005.

[14] S. Takahashi, I. Fujishiro, Y. Takeshima, andT. Nishita. A feature-driven approach to locatingoptimal viewpoints for volume. InProceedings ofIEEE Visualization’05, pages 495–502, 2005.

[15] J. van Wijk. The value of visualization. InPro-ceedings of IEEE Visualization’05, pages 567–574, 2005.

[16] J. van Wijk and W. Nuij. Smooth and efficientzooming and panning. InProceedings of IEEE In-foVis ’03, pages 15–22, 2003.

[17] J. van Wijk and W. Nuij. A model for smoothviewing and navigation of large 2d informationspaces. IEEE Transactions on Visualization andComputer Graphics, 10(4):447–458, 2004.

[18] P. P. Vazquez, M. Feixas, M. Sbert, and W. Hei-drich. Viewpoint selection using viewpoint en-tropy. InProceedings of VMV ’01, pages 273–280,2001.

[19] P. P. Vazquez, M. Feixas, M. Sbert, and W. Hei-drich. Automatic view selection using viewpointentropy and its application to image-based model-ing. Computer Graphics Forum, 22(5):689–700,2003.

[20] P. P. Vazquez, M. Feixas, M. Sbert, and A. Llo-bet. A new tool for obtaining good views formolecules. InProceedings of VisSym ’02, pages183–188, 2002.

[21] I. Viola, M. E. Groller, K. Buhler, M. Had-wiger, B. Preim, D. Ebert, M.C. Sousa,and D. Stredney. Illustrative visualiza-tion. Visualization 2005 Tutorial, 2005.http://www.cg.tuwien.ac.at/research/vis/exvisation/vis-tutorial/.

[22] I. Viola, A. Kanitsar, and M. E. Groller.Importance-driven volume rendering. InProceed-ings of IEEE Visualization’04, pages 139–145,2004.

[23] I. Viola, A. Kanitsar, and M. E. Groller.Importance-driven feature enhancement in volumevisualization.IEEE Transactions on Visualizationand Computer Graphics, 11(4):408–418, 2005.

[24] Volume data set repository, Vi-enna University of Technology,http://www.cg.tuwien.ac.at/research/vis/datasets/,2006.

Appendix

Information-Theoretic Measures:Let X be a finite set, letX be a random variable tak-ing valuesx in X with distribution p(x) = Pr[X = x].Likewise, letY be a random variable taking valuesy inY . TheShannon entropy H(X) of a random variableXis defined by

H(X) = − ∑x∈X

p(x) logp(x). (8)

The Shannon entropyH(X), also denoted byH(p),measures the average uncertainty of random variableX .All logarithms are base 2 and entropy is expressed inbits. The convention that 0 log0= 0 is used. Thecondi-tional entropy is defined by

H(Y |X) = − ∑x∈X

p(x) ∑y∈Y

p(y|x) logp(y|x), (9)

wherep(y|x) = Pr[Y = y|X = x] is the conditional prob-ability. The conditional entropyH(Y |X) measures the

12 REFERENCES

average uncertainty associated withY if we know theoutcome ofX . In general,H(Y |X) 6= H(X |Y ), andH(X) ≥ H(X |Y ) ≥ 0.Themutual information betweenX andY is defined by

I(X ,Y ) = H(X)−H(X |Y) = H(Y )−H(Y |X)

= ∑x∈X

p(x) ∑y∈Y

p(y|x) logp(y|x)p(y)

. (10)

The mutual informationI(X ,Y ) is a measure of theshared information betweenX andY . It can be seenthatI(X ,Y ) = I(Y,X) ≥ 0.The relative entropy or Kullback-Leibler distance be-tween two probability distributionsp andq is definedas

KL(p|q) = ∑x∈X

p(x) logp(x)q(x)

, (11)

where, from continuity, we use the convention that0 log0= 0, p(x) log p(x)

0 = ∞ if p(x) > 0 and 0log00 = 0.

The relative entropyKL(p|q) is a measure of the inef-ficiency of assuming that the distribution isq when thetrue distribution isp.The Jensen-Shannon divergence betweenn probabil-ity distributionsp1, p2, . . . , pn, with their correspondingweightsπ1,π2, . . . ,πn fulfilling ∑n

i=1πi = 1, is definedby

JS(p1, p2, . . . , pn) = H(n

∑i=1

πipi)−n

∑i=1

πiH(pi). (12)

The Jensen-Shannon divergence measures howfar arethe probabilitiespi from their mixture∑n

i=1 πi pi. Itequals zero if and only if all thepi are equal.

Information-Theoretic Viewpoint Quality Mea-sures for Polygonal Data:Viewpoint entropy measure based on Shannon entropy(Equation 8) is defined as

Hv = −

N f

∑i=0

ai

atlog

ai

at, (13)

whereN f is the number of polygons of the scene,ai isthe projected area of polygoni over the sphere of direc-tions centered at viewpointv, a0 represents the projected

area of background in open scenes, andat = ∑N fi=0ai is

the total area of the sphere. The maximum entropy isobtained when a certain viewpoint can see all the poly-gons with the same projected areaai.Viewpoint measure based on Kullback-Leibler distance(Equation 11) is defined by

KLv =

N f

∑i=1

ai

atlog

aiatAiAT

, (14)

whereai is the projected area of polygoni, at = ∑N fi=1 ai,

Ai is the actual area of polygoni andAT = ∑N fi=1 Ai is the

total area of the scene or object.