Information Visualization Engine (iVE) by PETER JOHN VINTON B.S., University of California, 1985 A thesis submitted to the Graduate Faculty of the University of Colorado at Colorado Springs in partial fulfillment of the requirements for the degree of Masters of Science Department of Computer Science 2004
99
Embed
Master Thesis Proposalcs.uccs.edu/~gsc/pub/master/pjvinton/doc/thesis.doc · Web viewA thesis submitted to the Graduate Faculty of the. University of Colorado at Colorado Springs.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Information Visualization Engine
(iVE)
by
PETER JOHN VINTON
B.S., University of California, 1985
A thesis submitted to the Graduate Faculty of the
University of Colorado at Colorado Springs
in partial fulfillment of the
requirements for the degree of
Masters of Science
Department of Computer Science
2004
This thesis for Master of Science degree byPeter John Vinton
Figure 2 is the Main GUI that appears once the iVE application is started.
Figure 3 iVE Tutorial
Figure 3 is the Tutorial GUI that appears when the Tutorial Button in the Main
GUI is clicked. The information in this GUI should be read before submitting a search in
order to understand the results of the iVE visualization.
Figure 4 Actual iVE Results
This figure illustrates the results of an actual iVE query. This particular GUI was
the result of entering the search phrase machu picchu hiking. There is a large amount of
perceptual information. The perceptual information is quite evident; look for the result
that has the most dots on or near the outer circle. There is also significant cognitive
information upon further examination. The cognitive information can be perceived by
examining which type of dot is closest to the outer circle (each dot represents a different
feature of the document as stated in the Figure 1 iVE Tutorial).
Figures 5 through 7, Least, Median, and Most Relevant iVE Results and their
corresponding URL documents, respectively:
The next set of figures shows what iVE considers the least relevant URL
document (figure 5), the median relevant URL document (figure 6), and the most relevant
URL documents (figure 7). The individual visualizations were cropped from figure 4
Actual iVE Results. The URL document below the visualization is the corresponding
URL document activated by clicking on the associated URL button. There is some
subjectivity as to exactly which of most relevant documents is the most relevant, but
there is a definite objective difference between the least relevant documents and the most
relevant documents.
37
Figure 2 iVE Main GUI, appears when iVE application is started
38
Figure 3 iVE Tutorial, activated by the Tutorial Button in the Main GUI
39
Figure 4 Actual iVE Results, resulting from entering the search term machu picchu hiking in the Main GUI
40
Figure 4 cont.
41
Figure 5 A Least Relevant Visualization and Associated URL Document, this visualization is cropped from Figure 4 Actual iVE Results, third from the bottom
42
Figure 6 A Median Relevant Visualization and Associated URL Document, this visualization is cropped from Figure 4 Actual iVE Results, sixth from the top
43
Figure 6 cont.
44
Figure 7 A Most Relevant Visualization and Associated URL Document, this visualization is cropped from Figure 4 Actual iVE Results, second from the bottom
45
Figure 7 cont.
46
Figure 7 cont.
47
Figure 7 cont.
48
Figure 7 cont.
49
Chapter 5
iVE Evaluation
Usability Evaluation Design Background
Usability evaluation for GUIs and websites are somewhat standardized. GUIs and
websites are usually evaluated in regard to some task being performed. For example
people intending or planning to buy a book first search for a book, review the results of
the search, and then select and purchase the book. Each of these actions can have many
levels or steps.
Looking at just the purchasing step, a typical process is for the customer to first
select a book, add it to some sort of shopping cart, check out, go to a purchasing form,
and then confirm the purchase. There is usually a separate GUI associated with each of
these sub-steps. As you can see, from the viewpoint of website/GUI design, the overall
accomplishment of a task is quite involved.
In this task based domain the items usually evaluated are: ease of navigating
through the website, ability to redo/undo actions, smooth flow from GUI to GUI,
knowledge of relative location within the website21 etc. However, in regards to
information visualization, there is currently no standardized usability testing22 and
usability testing in this domain is considered an ad-hoc process23. This is due to several
reasons. First information visualization is still a very young area of research. During the
years of 1995-1997 only 6% of the papers about information visualization dealt with
usability testing24. Also how a person interprets a visualization relies on factors such as
their socio-demographic profile, cognitive abilities, competency in understanding the
visualization, knowledge base25, and amount of training26, etc. Trying to take all these
items into account in the form of a standardized usability test is a daunting task and has
yet to be done.
Although there is no standardized usability testing for information visualization,
some papers have categorized and structured testing into certain groups.
One paper has categorized usability testing for information visualization into 3 main
groups27.
1. visual representation usability, referring to the expressiveness and quality of the
resulting image.
2. interface usability, related to the set of interaction mechanisms provided to users
so they can interact with the data through the visualization
3. data usability, devoted mainly to the quality of data for supporting users’ tasks
In addition a recent paper has broken down testing of information visualization into 4
types of experiments/studies28:
1. Controlled experiments comparing design elements such as specific widgets in a
GUI such as scroll bars or alpha sliders or types of buttons, etc.
2. Usability evaluation of a tool by itself where feedback is provided by users.
3. Controlled experiments consisting of comparing two or more tools such as
comparing types of tree visualization tools.
51
4. Case studies of tools in a realistic setting where users are able to spend a lot of
time with a tool in an actual environment of use (i.e. not just in a lab) and then
give feedback.
There are also some general usability testing concepts:
First tests should be made simple enough to be accomplished is a short period of time
and the items or questions in the test need to be specific enough to measure performance.
The same input data should be used for each evaluation to provide some uniformity and
standardization29 30.
Second, a captive audience should be used31. In general an online survey form is not
a good idea since people will sometimes fill out the survey before using the website/GUI
or, after using the website/GUI, will often leave the website/GUI without filling out the
survey.
Third the evaluator performing the test should watch what the people actually do
rather than rely on their feedback after the test32. This is because people in general will
tend to bend the truth toward what they think the evaluator wants to hear. Also in telling
the evaluator what they did people are actually telling the evaluator what they remember
doing. Finally people will rationalize their actions when all that is known for certain is
what they actually did.
Taking all the above factors into account any statistical results from a usability test
should be taken with caution.
It should also be noted that iVE focuses on a very specific and narrow segment of the
information retrieval task, it is in a sense a one-two punch in contrast with a whole 15-
round fight of the information retrieval task.
52
Evaluation Design
Taking all these factors into account a usability test for iVE has been tailored.
First no question is directly asked to participants in the evaluation. Second, observations
are recorded on how the participants respond and interact with the visualization tools.
And third an evaluation sheet is given to the participants after they have used the tool.
The evaluation questions consist of a combination of two experiment categories, one
category is the usability evaluation of the tool and the other category is a controlled
experiment of comparing iVE to another tool (www.kartoo.com). All users give each
tool the same input and there are only 10 questions on the evaluation sheet to keep the
evaluation short and hopefully concise.
53
Evaluation Results
The actual evaluation form can be seen in the next section. The statistical data
collected between iVE and kartoo is listed below:
Rating Scale: 5 – good / works well……………..1 – needs work
iVE kartoo
1. Tutorial: 4.00 4.42
2. Visual Organization: 4.00 4.54
3. Meaningful Use of Color: 3.82 4.00
4. Data Interaction: 3.64 4.64
5. Learning Curve: 3.91 4.63
6. Visualization Content: 3.36 4.55
7. Visualization Comparison: 3.81 3.72
8. Use in Other Domains: 3.72 4.09
9. Speed of Use: 3.63 3.91
10. Overall Functionality: 3.91 3.91
The were 11 participants in this study and all the participants were college
students.
As you can tell from the statistical data, the participants preferred kartoo in almost
every category, but there appears to be a contradiction between liking the tool versus the
functionality of the tool. This is evident by the ‘10. Overall Functionality’ rating of iVE
and kartoo being equal. The other category where participants preferred iVE was ‘7.
Visualization Comparison’. Both of these categories are more functional in nature.
54
The general comments noted about kartoo were: “…it was more interactive”, “…
the background coloring is better”, “…it was more fun to use” (because of its animation).
The general comments noted about iVE were: “…it was more old-school” and “…it was
more precise”.
Overall, in this demographic, the entertainment factor appears to have been a
determining factor in rating a visualization tool. And again it should be noted that
statistical information about information visualization testing should be taken with
caution.
Observations of the participants using iVE revealed the following functional
deficiencies: the scrolling resolution was too course, the tutorial would benefit with some
simple pictorial comparisons at the beginning, participants often hit the enter key on the
keyboard before having to click on the submit button, and the participants preferred
having all the visualizations visible in one window instead of having to scroll in the
window to see all the visualizations.
55
Evaluation Form
Comparison / Usability Evaluation
RATING SCALE: 5 – GOOD / WORKS WELL……………..1 – NEEDS WORK
1. Tutorial: Was the tutorial or help file understandable / informative20p229?
iVE Rating: __________ kartoo Rating: __________
2. Visual Organization: Were the visualizations presented in an orderly manner and were the visualizations evident (one visualization did not hide or obscure the another) 18slide10?
iVE Rating: __________ kartoo Rating: __________
3. Meaningful Use of Color: Did the colors help in interpreting/examining the visualization 18slide11?
iVE Rating: __________ kartoo Rating: __________
4. Data Interaction: Were you able to examine/interact with the data the visualization represented 15section1para4?
iVE Rating: __________ kartoo Rating: __________
5. Learning Curve: Was it easy to learn how to use the visualization tool?
iVE Rating: __________ kartoo Rating: __________
56
6. Visualization Content: By just looking at the visualization with no further action did the visualization provide useful characteristics of the document?
iVE Rating: __________ kartoo Rating: __________
7. Visualization Comparison: Did the visualization provide you with a means of comparing on document to another?
iVE Rating: __________ kartoo Rating: __________
8. Use in Other Domains: Based on the design of the visualization could it be applied to other areas, such as determining which photographs are more relevant, or which bacteria are the most relevant (probable cause), or which drugs are the most relevant (remedy for certain conditions)?
iVE Rating: __________ kartoo Rating: __________
9. Speed of Use: Was the image quick to interpret15section 2?
iVE Rating: __________ kartoo Rating: __________
10. Overall Functionality: Did the visualization tool save you time in determining which document was more relevant?
iVE Rating: __________ kartoo Rating: __________
57
Chapter 6
Conclusions and Future Research
Accomplishment
Based on the evaluation study of iVE it appears iVE has accomplished the task of
document relevancy through information visualization. iVE is quick to interpret, and
gives the user meaningful cognitive information. The user is able to quickly select and
view just the most relevant documents.
Problems Faced and Solutions
The following two standardization issues were encountered:
1. One problem encountered is URLs do not have a standardized format: some URLs
end in a '/', some do not, some start with 'www.', some do not …By analyzing the valid
URLs that did not appear in the visualization, it was apparent how the URL differed from
what was already being 'pattern-matched'. To compensate, another pattern was added for
matching (though there are probably more variations).
2. There are no standardized usability tests for information visualization, so though an
evaluation was performed, its statistical results should be viewed with some caution.
Improving iVE
Reviewing the results of the evaluation there are several areas iVE can improve
upon and several areas where there appears to have been some confusion. The apparent
existence of areas of confusion is based on the participants’ comments and actions.
The first items to be covered will be the areas of improvement followed by a
discussion of the areas of confusion. For clarity, each evaluation area is italicized and
followed by the corresponding question number used in the evaluation. In addition many
of these areas are somewhat coupled, and an improvement in one area may improve
another. For example, a better tutorial (1.)could improve the speed of use (9.)as well as
the learning curve (5.).
The tutorial (1.) and learning curve (5.) areas can be improved by putting the
visualization examples at the beginning of the tutorial instead of the end. Only two types
of visualizations should be shown, one visualization representing a relevant document
and the other visualization representing an irrelevant document. Several sets of these
visualizations could be shown to give the user a quick visualization training session.
The visual organization (2.) and speed of use (9.) can be improved by first
discarding the scrollbar paradigm in favor of displaying all the visualizations in one
window. A hierarchical ordering could be designed wherein the most important
visualization would reside on an inner delimiting ring and the visualizations of less
importance would reside on concentric outer delimiting rings. Also, the visualizations
59
near the center of this display would have a localized reddish background to key the
user’s eye in on the more relevant documents. Visualizations residing further from the
center would have colors gradated towards the blue end of the light spectrum to
downplay their relevancy. This design would also be in agreement with Fitt’s Law.
Furthermore, participants did not use the attribute of mentally creating a shape
based on the positions of the feature dots as a relevancy factor. This overall shape could
be made more explicit by enveloping all the indicator dots together in a single outline and
filling all the ‘non-feature dot’ areas within the outline with a color different than the
background. This would result in an image whose shape would be easier to view.
An additional enhancement to iVE, not directly related to the evaluation, would
be to make the visualizations expandable and collapsible so that more cognitive
information would be captured in the visualization. For example, a user would click on
one of the existing a high-level feature dots and this would bring up another visualization
of the same design that breaks down the selected feature into sub-features.
The first area of confusion to be discussed is visualization content (6.). The result
in this area directly conflicts with the area of visualization comparison (7.) where iVE
had a higher rating than kartoo. The question to pose is, “How can an item of less
visualization content produce better visualization comparison?” Due to this confusion an
improvement in this area may not be unnecessary.
The next two areas of confusion are data interaction (4.) and meaningful use of
color (3.). Based on the participants comments such as “…it was more interactive”, “…I
like the blue background color”, “…it was more fun to use”, and “…its just like neurons
firing” and based upon the participants action of moving the cursor around the kartoo
60
visualizations to view the animation, it appears that there was confusion between data
interaction vs. visualization animation/action and between meaningful use of color vs.
aesthetically pleasing color. Again due to this confusion, it may not be necessary to
change iVE in these areas.
In the area of use in other domains (8.) no comments or actions were observed.
But due to the characteristic that ‘if features in other domains are measurable or
quantifiable, iVE can visualize them’ and due to the fact that iVE is already better in the
area of visualization comparison, improvement in this area may not be necessary.
A drawback to iVE is that it has only been tested in the domain of the written word.
iVE should theoretically work in any domain but further testing is needed for
confirmation. A challenge exists in finding meaningful features to extract for the other
domains.
Impact of Research
The impact of this research is that iVE can significantly reduce the time a user
spends searching for relevant information. There could also be the side effect that the
overall shape of the iVE visualization could reveal other qualities/groupings of
information units just like molecules with similar shapes having similar properties.
61
References
CHAPTER 2
1 R. Chimera, May 1992 “Value Bars: An Information Visualization and Navigation
Tool of Multi-attribute Listings", In Proceedings of the SCM SIGCHI Conference on
Human Factors in Computing Systems, pp. 293-294
2 G. G. Robertson, S. K. Card, J. D. Mackinlay, "Information visualization using 3D
interactive animation”, Edited by Paul S. Jacobs, publisher Lawrence Erlbaum
Associates, ISBN 0-8058-1189-3.
3 M. A. Hearst, 1995, “TileBars: Visualization of terms distribution information in full-
length document access", Association for Computing Machinery
4 G. G. Robertson, J. D. Mackinlay, S. K. Card, 1991, "Cone Trees: Animated 3D
visualizations of hierarchical information", Proceedings of SIGCHI'91, pp. 189-194
5 Mackinlay J.D., Robertson G.G., Card S.K., 1991, "The Perspective Wall: Detail and
context smoothly integrated", In Proceedings of SIGCHI'91, pp. 173 179
6 G. W. Furnas, 1986, "Generalized Fisheye Views", Proceedings of SIGCHI'86, pp. 16-
23
7 S. E. Poltrock, G. W. Furnas, K. M. Fairchild, 1988, "SemNet: Three-Dimensional
Graphic Representations of Large Knowledge Bases", Cognitive Science and its
Application for Human-Computer Interface, R. Guindon (Editor), Lawrence Erlbaum,
New Jersey
8 R. Spence, M. Apperly, 1982, "Data base navigation: an office environment for the
professional", Behaviour and Information Technology 1, (1), 43-54
9 J. D. Mackinlay, G. G. Robertson, S. K. Card, The Information Visualizer: A 3D user
Interface for Information Retrieval, Xerox Palo Alto Research Center,