Information Visualization CSCI 6174: Open Problems in CS Fall 2011 Richard Fowler
Jan 11, 2016
Information Visualization
CSCI 6174: Open Problems in CS
Fall 2011
Richard Fowler
Ya gotta visualize …
• I see what you mean …
– so, visualization can be considered not just a visual process, but a cognitive (thought) process as well
• And a very large part of human brain taken up with visual system
– and that part of the brain is still useful beyond “simply” getting an image of the world
– … which is in fact pretty complicated
Visualization is …
• Visualize:– “To form a mental image or vision of …”– “To imagine or remember as if actually seeing …”– Firmly embedded in language, if you see what I mean
• (Computer-based) Visualization:– “The use of computer-supported, interactive, visual
representations of data to amplify cognition”• Cognition is the acquisition or use of knowledge• Card, Mackinlay Shneiderman ’98
– Scientific Visualization: physical
– Information Visualization: abstract
Visualization is not New
• Cave guys, prehistory, hunting
• Directions and maps
• Science and graphs– e.g, Boyle: p = vt
• … but, computer based visualization is new– … and the systematic delineation of the design
space of (especially information) visualization systems is growing nonlinearly
Visualization and Insight
• “Computing is about insight, not numbers”– Richard Hamming, 1969– And a lot of people knew that already
• Likewise, purpose of visualization is insight, not pictures– “An information visualization is a visual user
interface to information with the goal of providing insight.”, (Spence, in North)
• Goals of insight– Discovery– Explanation– Decision making
“Computing is about insight, not numbers”
• Numbers – states, %college, income:State % college degree income State % college degree income
“Computing is about insight, not numbers”
• Insights:– What state has highest income?, What is relation between education and income?, Any outliers?
State % college degree income State % college degree income
“Computing is about insight, not numbers”
• Insights:– What state has highest income?, What is relation between education and income?, Any
outliers?
Not about Useless Visual Stuff - Clutter
• “3d” adds nothing– (at best)
Detrimental useless stuff
• USA Today
An Example, Challenger Shuttle
• Presented to decision makers– To launch or not– Temp in 30’s
• “Chart junk”
• Finding form of visual representation is important– cf. “Many Eyes”
An Example, Challenger Shuttle
• With right visualization, insight (pattern) is obvious– Plot o-ring damage vs. temperature
Insight …
• Some examples ….
A Classic Static Graphics Example
• Napolean’s Russian campaign– N soldiers, distance, temperature – from Tufte
For what it’s worth …
• x
Visualization Pipeline:Mapping Data to Visual Form
• Visualizations: – “adjustable mappings from data to visual form to human perceiver”
• Series of data transformations– Multiple chained transformations– Human adjust the transformation
• Entire pipeline comprises an information visualization
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Visualization Stages
• Data transformations:– Map raw data (idiosynchratic form) into data tables (relational descriptions
including metatags)
• Visual Mappings:– Transform data tables into visual structures that combine spatial substrates,
marks, and graphical properties
• View Transformations:– Create views of the Visual Structures by specifying graphical parameters
such as position, scaling, and clipping
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Information Structure
• Visual mapping is starting point for visualization design
• Includes identifying underlying structure in data, and for display– Tabular structure– Spatial and temporal structure– Trees, networks, and graphs– Text and document collection structure– Combining multiple strategies
• Impacts how user thinks about problem - Mental model
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
A “Taxonomy” of Visualization
SpacePhysical Data1D, 2D, 3DMultiple Dimensions, >3TreesNetworks
InteractionDynamic QueriesInteractive AnalysisOverview + Detail
Focus + ContextFisheye ViewsBifocal LensDistorted ViewsAlternate Geometry
Data Mapping: TextText in 1DText in 2DText in 3DText in 3D + Time
Higher-Level VisualizationInfoSphereWorkspacesVisual Objects
1D Linear Data
1D Linear Data
2D Map Data
3D World Data
Multiple Dimensions > 3
• “Straightforward” 1, 2, 3 dimensional representations– E.g., time and
concrete
• Can extend to more challenging n-dimensional representations– Which is at core of
visualization challenges
• E.g., Feiner et al., “worlds within worlds”
Temporal Data
Trees, Networks, and Graphs
• Connections between /among individual entities
• Most generally, a graph is a set edges connected by a set of vertices– G = V(e)– “Most general” data
structure
• Graph layout and display an area of iv
• Trees, as data structure, occur … a lot– E.g., Cone trees
Tree/Hierarchical Data• Workspaces
– The Information Visualizer: An Information Workspace by G. R. Robertson, S. K. Card, J. M. Mackinlay, 1991 CACM
Networks
• E.g., network traffic data
• Visualization of NSFNET
• Cox, D. & Patterson, R., NCSA, 1992
• Routes of the Internet, 1/15/05
• The opte project
• Earlier snapshot in permanent collection of NY Museum of Modern Art
• 3-d hyperbolic tree of web sites using Prefuse
Abstract – Non-physical
• Concept map– Graph of
“conceptual” information
• From Berners-Lee’s proposal to CERN for what is now called www, March 1989
• Manual “graph drawing”
http://www.nic.funet.fi/index/FUNET/history/internet/w3c/proposal.html
FYI - Demo
• http://thejit.org/
Text and Document Collection Structure
• Derivation of relationships upon which display is to be based a challenge
• E.g., Wise et al
Text and Document Collection Structure, e.g., Galaxy of News
• x
Overview Strategies
• Typically useful, or critical, to have “feel” for all data– Then, allows closer inspection in “context” of all data– Overview + detail, focus + context
• Known from the outset of visualization– Bifocal Lens
• Database navigation: An Office Environment for the Professional by R. Spence and M. Apperley
• Shneiderman mantra– “overview first, zoom and filter, details on demand”
Focus+Context: Fisheye Views, 1
• Detail + Overview – Keep focus, while remaining aware
of context
• Fisheye views– Physical, of course, also ..– A distance function. (based on
relevance)– Given a target item (focus)– Less relevant other items are
dropped from the display– Classic cover
• New Yorker’s idea of the world
Focus+Context: Fisheye Views, 2• Detail + Overview
– Keep focus while remaining aware of context
• Fisheye views– Physical, of course, also ..– A distance function. (based on relevance)– Given a target item (focus)– Less relevant other items are dropped from
the display – Or, are just physically smaller – distortion
Focus + Context – Spatial Distortion
• Selectively reduce complexity as f(user’s viewpoint)
• Spatial distortion– Project network
on distorted space
• Viewing “lens”
Focus + Context – Spatial Distortion
• Selectively reduce complexity as f(user’s viewpoint)
• Spatial distortion– Project network
on distorted space
• Viewing “lens”
• Seamless transition
Focus + Context – Hyperbolic View
• Again, selectively reduce complexity as f(user’s viewpt.)• Smooth change during interaction
Focus + Context – Hyperbolic View
• Also, in 3 space
• Demo
• 3-d hyperbolic tree of web sites using Prefuse
Tools
IBM’s Many Eyes
• Multiple visualizations
IBM’s Many Eyes
• Visualization types
IBM’s Many Eyes
• Life expectancy vs. health care costs
• http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/life-expectancy-vs-per-capita-annu
Visualization Pipeline:Mapping Data to Visual Form
• Visualizations: – “adjustable mappings from data to visual form to human perceiver”
• Series of data transformations– Multiple chained transformations– Human adjust the transformation
• Entire pipeline comprises an information visualization
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Visualization Stages
• Data transformations:– Map raw data (idiosynchratic form) into data tables (relational descriptions
including metatags)
• Visual Mappings:– Transform data tables into visual structures that combine spatial substrates,
marks, and graphical properties
• View Transformations:– Create views of the Visual Structures by specifying graphical parameters
such as position, scaling, and clipping
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Information Structure
• Visual mapping is starting point for visualization design
• Includes identifying underlying structure in data, and for display– Tabular structure– Spatial and temporal structure– Trees, networks, and graphs– Text and document collection structure– Combining multiple strategies
• Impacts how user thinks about problem - Mental model
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Information Vis Systems at UTPA
Information Vis Systems at UTPA
• Data mining, VAS - Visual Analysis System - Hubs and authorities
• Text visualization, ATV - Abstract Text Viewer - Tag clouds and such
• Clinician’s tool for personality, DID-TM– Dissociative Identity Disorder – Trait Mapper, Visualizing personality
• Reading: Knowledge domain citation and semantic structure– Knowledge worker’s tool
– Selectively varying density in graph visualization
– Perceiving organization
• Reports available on web site
Data mining, VAS - Visual Analysis System
Data mining, VAS - Visual Analysis System
• Data mining, VAS - Visual Analysis System– Hubs and authorities
• Emphasizes effort on data– Collection and transformation to form dataset
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Data Mining: Hubs and Authorities
• Attempt to overcome shortcomings of text indexed search engines
• Graph and cluster based approach
• Link structure of WWW – “latent human annotation”
• Link to page implicit “endorsement” of page
• Web as directed graph
• Based on link structure, characterizes pages as:1. “Authorities”
- best sources of information- high indegree (refined)
2. “Hubs”- provide collections of links to authorities- high outdegree (refined)
The System
• Goal: Allow user to rapidly and incrementally assess utility of web pages– Data mining (hubs and authorities)– Visualization– Filtering – User interaction tools
System Architecture
W W W Search Engine
User Interact
User QueryQuery Results
Fetch Pages
Hub Scores
Layout Pages
Filter for Display
• Goal: Allow users to systematically and incrementally access web pages
W W W Search Engine
User Interact
User QueryQuery Results
Fetch Pages
Hub Scores
Layout Pages
Filter for Display
• Goal: Allow users to systematically and incrementally access web pages
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Example Screen
• Only pages of highest hub and authority scores
• Red: Hubs• Blue:Authorities• User can select
pages
Example Screen - Detail
ATV - Abstract Text ViewerText Visualization
Tag cloud from infovis wiki
ATV - Abstract Text ViewerText Visualization
• Electronic presentation of text for a generation– Ubiquitous
• Manuals, web document/pages, books, …
– Surprisingly few tools for augmenting
• ATV:– Text reading tool for
electronic documents– Uses well-known and
novel techniques
ATV Electronic Presentation Techniques
• Overview + Detail– Facilitates orientation and navigation– Works for spatial data and text
• Abstract text’s content and use to organize– Enhance reader’s efficiency and effectiveness– Use existing elements: HTML tags– Use system derived elements: keywords, …
Paragraph View
• ATV is a browser
• Left for structure & content, “overview”
• Right for enhanced text, “detail”
HTML Structure View
• Headings reveal structure (outline)
• Entire document available
Link View
• All links (navigation elements) available
Word Frequency View
• Crawler reads domain
• Words above threshold in domain listed
• Overview of domain
Word Frequency View
• Words with frequencies > 2 thresholds displayed
Detail (Text) Window
• Darkness of text = f(relatedness) to entire document
• Similarity of paragraph to entire document
Detail (Text) Window
• Word search provided
ATV Conclusions
• Testbed for implementing and testing text abstraction and viewing techniques
• Currently provides tools targeting HTML documents
• Extension to non-marked documents
• Platform for usability testing
DID-TM
Clinician’s tool for personality, DID-TM
• Clinician’s tool for personality, DID-TM– Dissociative Identity Disorder – Trait Mapper– Visualizing personality
• Dissociative Identity Disorder – Trait Mapper
• Tool for clinician use– Manage complexity of case history– Show visually state and progress of client in integrating identities
• Well known visualization techniques– E.g., parallel coordinates
• Novel techniques– E.g., coding of communication and shift over time
DID-TM
• Personality profiles• Identity communication graph• Stored and indexed clinician’s notes
Visualizing Knowledge Domain Structure
• Knowledge worker’s (or anyone’s) tool– Yet again, managing large amounts of information
• -Tools for organizing knowledge domain– E.g., scientist (or student) learning about a new domain– Become acquainted with literature or find new relations
and information– Citeseer
• Exploring and retrieving information– Visual representation of citation network– Visual representations of semantic similarity of documents– Similar to Document Explorer
Network Visualization
Visualizing Knowledge Domain Structure
• Exploring and retrieving information
– Visual representation of citation network• Relationships among documents as shown by citations (references)
– Visual representations of document semantic similarity network
• Semantic document network• Again, relations, now based on content similarity
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Extracting and Organizing Content
• Networks: 1. Citations form graph 2. Document similarity
– Word cooccurrence– Similarity of Documents
• Compare all pairs of documents• Use distance matrix to derive
network
• Network density varies
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Displaying the Networks:Node Positioning using Spring Layout
• Physical spring analog– “Spring Embedder” algorithm
• Can vary spring length, strength, elastic properties– E.g., document similarity
• Example at right in 3D• Interaction by movement
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
Network Visualization
• Visualizing Knowledge Domain Citation and Semantic Structure
• Citeseer Visualization
• 1,138 documents from Citeseer collection
• Citation network– Nodes are documents, links are citations (references)– Here, links are weighted by document similarity
Citeseer Visualization - Query• Query “information visualization”• Results used to form citation graph and visual representation
Citeseer Visualization – Results as MST
• Minimum cost spanning tree (graph) used to represent query results
Earlier Network Display & Interaction Tools
• Overviews– Nodes of highest degree– Landmarks: Visible, selectable
• Bookmarks– Set & return to viewpoint
• Fluid motion
• Network density selectable
• Anchors– User-defined selectable
• Signposts– Anchor labeled with overview
nodes– Global orientation at level of
local detail
• Expand and Collapse Nodes
• Color
Display & Interaction Tools, cont.
• Stereo Viewing
– LCD glasses
– Head tracked, “look around”
– Compromise immersion for text tasks
Selective Density in Network Viewing
Overview
• Reducing and managing network density for visualization– Varying structural density– Distorted space display techniques
• Deriving quantitative metrics from documents– From which document network created
• Pathfinder networks– Path length limited minimum cost networks
• A new hybrid representation to selectively vary density
• Internet alliances and partnershships, 2002
• www.orgnet.com/netindustry.html
• Trade relationships, 1992
• www.
Reducing and Managing Density
• Focus + context techniques “selectively” reduce density– User’s view affects display– Spatial distortion techniques
• Use same network, but change space upon which it is projected– Change network structure itself, depending on where focus is
• E.g., Furnas’ 1986 account of fevs, display as f(distance from focus)
• Threshold techniques – Display only links with weights > some value
• As part of structure derivation (network formation)– E.g., minimum cost spanning tree (MCST)
• Limiting case for connected graph– Pathfinder networks
Focus + Context – Hyperbolic View
• Again, selectively reduce complexity as f(user’s viewpt.)• Smooth change during interaction
Threshold
• Reduce complexity by eliminating links < some threshold– Not necessarily preserve connectivity
http://www.g3tvu.co.uk/Network,_Radio_Link_and_Route_Styles.htm
Varying and Reducing Density
• As shown, can vary display space and locally (selectively) reduce density– Distortion techniques
• Also, can reduce density globally (overall) – Link weight threshold, as shown
– Minimum spanning trees
– Here, Pathfinder networks
• Goal of work is to create representation that uses structural (vs. display) manipulation to provide global context and local detail
PfNets – Path Length Limited
• For some data set of distances– Here, data are provided by human subjects– Document network uses interdocument distances
• Construct network that is sufficiently dense that any node can be reached from any other node in q links
• q = n-1
Schvaneveldt et al., 1989
PfNets – Path Length Limited
• Smaller q– Denser graph
Schvaneveldt et al., 1989
Graph Display Considerations
• Graph display issues critical in visualization– And a field in itself
• Force directed layout– Widely used
• E.g., prefuse
• Works well for sparse graphs– Shows global relations well– Not so well for dense
PfNets forGlobal Context and Local Detail
• Combine sparse pfnet (inf, n-1) with more dense at point of interest
• Provide detail + context
Hybrid Pfnets
• Sparse– overv
iew.TIF
Hybrid Pfnets
• Dense
Hybrid Pfnets
• Dense zoomed in
Hybrid Pfnets
• Combined
Perception of Organization
Perception of Organization
• Self organizing systems
• Simple rules, complex behaviors
• Social insects– Ants, bees
• Flocks of birds– Fairly well modeled with few constraints– Coherence (cohesion) of flock– Distance from another individual– Direction
• What are the roles of the elements of organization used by people?
Perception of Organization
– Coherence (cohesion) of flock - Distance from another individual– Direction - Stereoscopy
Perception of Organization
– Coherence (cohesion) of flock - Distance from another individual– Direction - Stereoscopy
BONUS!
• Immersive interfaces, prescence, …
• New research effort
Introduction
• The “best” interfaces, and all systems, typically find their task utility through engagement (etc.) appropriate for the task
– This idea is at the core of arguments for the use of direct manipulation interfaces
• All of the following are interrelated:– Immersion, engagement, presence, virtual reality– 3D display and interaction devices
• In field of CS and HCI: “spatial interfaces”
• Will introduce the idea of presence
Immersion, “Virtual Reality”, and Virtual Environments
• Immersive interfaces– High sensory immersion – visual, auditory, haptic, proprioceptive
• “Virtual reality”, or, virtual environments– “Virtual reality is a technology that is used to generate a simulated environment
in digital form... Using the equipment, users are immersed in a totally virtual world.”
– Working definition – an immersive interactive system
• In context of “virtual reality”, immersion usually = spatial immersion
• Note: “Immersion” (and engagement and presence) is a continuum
– Text ... Visual and 3d .. Stereo ... HMD… “jacked in”– Cyberspace
• Term coined by Gibson in Neuromancer• … and in the 21st century, the Matrix
Immersion and Virtual Reality• “The mind has a strong desire to believe that the world it perceives
is real” – Jaron Lanier, among others
• For example, “illusion” (perception) of depth (for spatial immersion)• Stereo parallax• Head motion parallax• Object motion parallax• Texture scale
• Interaction: grab and move an object
• Proprioceptive cues: – when you reach out and see a hand where you believe your hand to be,
you accept the hand as your own
• Often you will accept what you see as “real” even if graphics poor
• Constellation of cues
Presence “The Aesthetic Impression of 3D Space”
• Sense of presence – Vividly 3d– Actually present in the world– Sense of being there– Holodeck …
• Presence has to do as much with engagement, as visual information– E.g., one can be “in the world”, when reading– Here, one sees, or visualizes, the world
• 3D depth cues are those elements that enhance feeling of 3 (vs. 2) dimensions in a display, – From occlusion to stereoscopic display
Presence “The Aesthetic Impression of 3D Space”
• Immersive interfaces– term used to describe interfaces/devices which lead toward immersion
(sense of presence, engagement) in the virtual environment presented on the display
• Virtual reality interfaces– term used similarly to immersive interfaces
• Degree of immersion– conventional desktop screen– fishtank virtual reality (semi-immersive workbench)– immersive virtual reality– augmented reality with video or optical blending– … number of cues …
Immersive and 3D Interfaces• Teleoperation
• Virtual and augmented reality
• Immersion and VR – contribution of components …
• Survey of 3D displays– Surround screen displays - CAVE– Input devices - Data glove– Data walls– Workbenches– Hemispherical display– Head-mounted displays– Arm-mounted displays– Virtual retinal display– Autostereoscopic displays
Sutherland’s 1960’s equipment• Ultimate display, 1965
• Sword of Damocles – 1st HMD– Actual camera-like metal shutters
Virtual and Augmented Reality
• Augmented reality shows real world with an overlay of additional overlay
• Knowlton (1975)
• Partially-silvered mirror over keyboard
• Programmable labels
• Tactile feedback
Augmented Reality, 2
• Enables users to see real world with an overlay of additional interaction– Situational awareness
• See through glasses
• Typically, add text+images to real world
• Very sensitive to head tracking, when used
Surround-screen displays
• Pro• less obtrusive headgear• multi-user?• better stereo• Con• occlusion problem• missing sides
Surround screen displays – CAVE, 1
• A room with walls and/or floor formed by rear projection screens– Head tracking– Stereo– Light scattering
problems
• Visual immersion– Field of view is
100% possible, ~200 degrees
Surround screen displays – CAVE, 2
• Typical size: 10’ x 10’ x 10’ room
• 2 or 3 walls are rear projection screens– Floor is projected from above
• User is – tracked – He/she also wears stereo shutter goggles…– Uses “wand” to manipulate
• Projects 3D scenes for viewer’s point of view on walls
– Walls vanish, user perceives full 3D scene– So, view is only correct for that viewer
• Cost is fairly high
UTPA Immersive Systems Lab~Summer, 2012
Proj.
Proj. Proj.
27’13’
Security area
21’
6’
TV
stor
age
~8’
CAVE
Ph
ysio
log
ica
lM
easu
rem
ent
E
qu
ipm
ent
Computers
Front Projection Screen
Proj.
Develo
pmen
t
Questions?
• .