Flexible Visual Authoring Using Operation History Sara Su Massachusetts Institute of Technology l April 8, 2009 Committee in Charge: Prof. Frédo Durand (MIT, Supervisor) Prof. Maneesh Agrawala (UC Berkeley) Prof. Robert C. Miller (MIT) Dr. Sylvain Paris (Adobe)
50
Embed
Flexible Visual Authoring Using Operation History · Flexible Visual Authoring Using Operation History Sara Su Massachusetts Institute of Technology ... Microsoft PowerPoint - defense.ppt
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Flexible Visual Authoring Using Operation HistorySara SuMassachusetts Institute of Technology
lApril 8, 2009
Committee in Charge:Prof. Frédo Durand (MIT, Supervisor)Prof. Maneesh Agrawala (UC Berkeley)Prof. Robert C. Miller (MIT)Dr. Sylvain Paris (Adobe)
System activity logs, instrumentation (not our focus)Operation history, undoVersion controlTutorials
Uses of selections and grouping
3
Uses of selections and groupingEfficient editing of sets of items (multiple selections)Hierarchical modeling, CAD
MotivationAddress limitations of standard techniques
Undo - sequentialSelections - not persistentGrouping – rigid structure expensive to modify
Thesis:
4
Thesis:Reusing operations, selections, and groups from a document’s
history can improve interaction for the end user.
Enhancing authoring and review
Visualizing history for non-linear interactionStoryboards: Interactive Visual Histories
Reusing complex selections for efficiencyHistory-Based Selection Expansion
Enabling bookmarking for flexible groupingSoft Groups: Multiple Selection Authoring and Reuse
5user operations
use
r se
lect
ions
Storyboards
SelectionExpansion
SoftGroups
Thesis contextDemonstrate techniques in context of visual authoring
Features in Inkscape vector graphics editor
Human componentEvaluations with beginner- and intermediate-level usersIterative design
6
Talk outlineInteractive Storyboards
Visualizing history for non-linear interaction
Selection ExpansionReusing complex selections for efficiency
Soft Groups
7
Soft GroupsFlexible authoring of multiple selections
Interactive StoryboardsVisualizing history for non-linear interaction
Selection Expansion
S ft G
8
Soft Groups
Motivation: Visual historiesEnable flexible browsing of history
Design a more intuitive interface to document’s editing historyShow history in spatial context
Enable flexible manipulation of historyInterface to selective undo
9
Related work: Undo
UndoRevisit historyUndo arbitrarily far backSequential
Selective undoText
[Kawasaki and Igarashi 2004]
10
TextSpreadsheetsGraphics
Amulet [Myers et al. 1997]
Related work: Graphical histories
Snapshots Editable graphical histories
11
[Meng et al. 1998][Kurlander and Feiner 1990]
[Agrawala et al. 2003][Goldman et al. 2006]
Film and schematic storyboardsAssembly diagrams
Our storyboard visualizationGraphically represents user editing actions
Assembly instructions for a document
Shows actions in context: action depictionsMust be descriptive, intuitive, and easy to select
12
13
Our storyboard visualizationGraphically represent user editing actions
Show actions in context: action depictions
Design considerationsDiscrete eventsB f d ft
14
Before and afterIn-place visualizationSummarization Figure-ground separation
ApplicationsSelective undo
User selects an action to undoConsider all later actions on the same objectCancel only those that are dependent
Spatial transforms: {translate, rotate}Appearance changes: {fill change, stroke change}Shape modifications: {scale, control point edit}
15
Alice
Bob
Depiction Key
Collaborative editing“Track changes”Asynchronous editing by multiple users
EvaluationGoals
Record users’ impressions after using storyboards for one hourEvaluate selective undo interface
Design12 beginner-level users of 2D drawing programsBackground interview, interactive tutorialR t “t i l” d i
16
Recreate a “typical” drawing
User feedbackStrengths
Free experimentationSpatial memory cuesPersistent history
LimitationsClutter, scalability
Addressing clutterPer-object history
“Magic lens” limits storyboard view
Multi-frame storyboardM lti l f i t b d
17
Multiple frames in a storyboard Multiple actions per frame
Summary: Interactive StoryboardsInteractive storyboards for visualizing history
Browsing history in spatial context
Composite, per-object, and multi-frame storyboardsSelective undo application
Collaborative editing
18
Selection ExpansionReusing complex selections for efficiency
Interactive Storyboards
S ft G
19
Soft Groups
Motivation: Selection reuseMultiple selections are fundamental in editing
Edit the same set of objects togetherReselecting the set can be repetitive, laborious
Esp. with overlapping, occluding objects
GroupsIntuitive, easy to build hierarchy
20
Intuitive, easy to build hierarchyAn item cannot belong to more than one group at a timeUngrouping/regrouping expensive
Related work: Selecting content
Transparency filters Multiblending [Baudisch and Gutwin 2004]Context-aware free-space transparency [Ishak and Feiner 2004]
Physical interaction metaphors“Paper peeling” windows [Beaudoin-Lafon 2001]Exposé [Apple 2003]
21
Exposé [Apple 2003]
Splatter [Ramos et al. 2006]Magic Lens [Bier et al. 1993]
Related work: Complex selections
Generalizing selectionsSelection guessing [Miller and Myers 2002]Selection classifier [Ritter and Basu 2009]Interactive query relaxation [Heer et al. 2008]
22
Related work: Adapting user interfaces
Resize/rearrange menus to reduce target acquisition timeFisheye menus [Bederson 2000]Flexcel [Thomas and Krogsæter 1993]
Dynamically organizing menu items – frequency, recency[Greenberg and Witten 1985][Mitchell and Shneiderman 1989]
23
[Mitchell and Shneiderman 1989]Split menus [Sears and Shneiderman 1994]
Selection expansionHypothesis: Items that have been edited together are likely to be edited
together again.
From an initial selection, expand to a larger set
B th i f f
24
Base the expansion on frequency of use
Greedy expansion strategyUser makes a selection (query)
Look in operation history for single best item to addCandidates = items that have been edited with the query setPick the item appearing most frequently
Expand the selection by one
25
Expand the selection by one
A simple example
Excerpt: Operations affecting {e}:
User’s initial selection is {e}
Compressing the matrix:
26
Query = {e}
Candidate object d:
Candidate object f:
Frequency = 5
Frequency = 2
27
Candidate object g:
q y
Frequency = 4
Query = {e,d}
Candidate object f:
Candidate object g:
Frequency = 2
Frequency = 2
28
q y
Query = {e,d,f}
Candidate object g:
Frequency = 2
Three expansions:
{e} {e,d} {e,d,f} {e,d,f,g}
29
Larger expansion steps
For efficiency, merge steps when we canLook for plateaus in maximum selection frequency
Recruited from general populationAll familiar with at least one 2D drawing program (not Inkscape)
ApparatusControlled lab settingModified version of Inkscape
32
Two-part study1. Selection reuse with existing histories
Evaluate how QuickSelect affects selection speed and accuracy
2. Selection reuse in free drawingRecord users’ subjective preferences in unconstrained drawing
Study 1: Existing historiesTwo conditions: standard selection, QuickSelect20 tasks: edit existing drawings
Procedure:
33
Hypothesis: QuickSelect will reduce time to complete the trials and reduce number of editing errors.
Results of Study 1
34
Increasing complexity of task
Study 2: Free drawingTry selections in a more realistic setting
ProcedureRecreate “typical” drawing described during interviewUnstructured drawing with prompts to try different selectionsNo measure of success
35
FeedbackEasy to learn and usePerceived improvement in speedPerceived improvement in accuracyStudy 2 more convincing about applicability
ObservationsStrengths of QuickSelect
Performance savings larger for more complex designsRe-selecting occluded contentRe-selecting objects of differing size
36
LimitationsPredictability and error handlingCombining selection toolsAdditional expansion heuristics
Summary: Selection ExpansionReuse of multiple selectionsSimple yet effective history-based strategy
Easy to learn and applySelection reuse can increase efficiency
Greater savings for more complex designs
37
Expansion behavior can be difficult to predict soft groups
Soft Groups
Selection Expansion
Interactive Storyboards
38
Soft GroupsFlexible authoring of multiple selections
Motivation: Flexible groupingGroups
Easy to use, membership in only one group at a time
SelectionsMembership created as needed, ephemeral
Selection expansionReuse selections from history, lacks predictability
39
Related workSelecting, grouping, taggingFlexible grouping - ScanScribe [Saund et al. 2003]Relation building from history [Pedersen and McDonald 2008]
Soft groupsUsers bookmark multiple selections they wish to reuseLike groups, soft groups are persistent and reusable
An item can belong to more than one soft group
Like selections, soft groups appear on demand
40
Retrieval interaction similar to selection expansionExpansion steps determined by user
Create Soft Group
Group creation
41
Create Soft Group
2
Group creation
42
Retrieve Soft Group
Group retrieval
43
Exploratory evaluationGoals
Get user feedback on ease of learning and useCompare soft groups to standard selection and groupingCompare soft groups to selection expansion
Nine beginner- to intermediate-level users of 2D software
44
Procedure:Recreate “typical” drawing described during interviewUnstructured drawing with no measure of successFirst, asked to try soft groupsSecond, introduced to selection expansion
Observations from user studyStrengths of soft groups
Straightforward use, easy to learnSpatial memory cues: “visual reminder”Improves efficiency of authoring
Fixed cost to creating soft groups but faster retrieval
Limitations
45
Error handlingVisibility, responsiveness
Comparison to selection expansionQuickSelect “seems faster” than soft groupsIntermediate users concerned about cost of correcting QSSG offer more control
Summary: Soft GroupsBookmarking selections for reuseComplementary alternative to standard selection and groupingPersistent like standard groupsAppear on demand like standard selections
Easy to learn and use
46
Users preference divided by experienceBeginners: efficiency of selection expansionIntermediate-level users: control of soft groups
Summary of thesis contributionsPresented three uses of history for the end user
Interactive StoryboardsSelection Expansion (QuickSelect)Soft Groups
Demonstrated in the context of vector graphics editing
47
User evaluations suggest increased efficiency and flexibility in editing
Applications and open challengesPrototyping
Selection reuse for faster prototyping and testing of variationsStoryboards lower the cost of experimentation
CollaborationRecorded history for collaborators
EducationStoryboards as tutorials
48
Storyboards as tutorials
Future workOther domainsExpert users
Longer-term observationKeystroke-level modeling
ConclusionsBigger picture: Mining operation history to enhance HCI
Demonstrated history-based techniques for improving authoring and review processes
49
AcknowledgmentsProf. Frédo DurandProf. Maneesh AgrawalaProf. Rob MillerDr. Sylvain Paris
Adobe Systems: Fred Aliaga, Steve Johnson, Craig Scull
50
Inkscape, Open Clip Art LibraryMIT Computer Graphics GroupSu Family
NSF Graduate Research Fellowship ProgramMIT Presidential Fellowship Program