5-1 5. Computer Science and Human-Computer Interaction 5.1. Motivation 5.2. User Interface Software Concepts 5.2.1. Design Process for Interactive Systems 5.2.2. Interaction Techniques 5.2.3. Interaction Styles 5.2.4. User Interface Management Systems 5.2.5. User Interface Description Languages 5.3. User Interface Software Tools 5.3.1. Window Systems 5.3.2. Toolkits 5.3.3. UIMS 5.4. Future Directions
38
Embed
5-1 5. Computer Science and Human-Computer Interaction 5.1. Motivation 5.2. User Interface Software Concepts 5.2.1. Design Process for Interactive Systems.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
5-1
5. Computer Science and Human-Computer Interaction
5.1. Motivation 5.2. User Interface Software Concepts
5.2.1. Design Process for Interactive Systems 5.2.2. Interaction Techniques 5.2.3. Interaction Styles 5.2.4. User Interface Management Systems 5.2.5. User Interface Description Languages
5.3. User Interface Software Tools 5.3.1. Window Systems 5.3.2. Toolkits 5.3.3. UIMS
5.4. Future Directions
5-2
5.1. Motivation
Easier to use, but harder to program [Myers] Multiprogramming Real-time programming Robustness Testability Modularization
Need to support iterative design and testing Software tools
5-3
5.2. User Interface Software Concepts
Dialogue independence Interface (syntax) vs. application (semantics, business
Current state of practice, use GUI techniques to specify GUI Plug together components with pre-defined interactive behavior
Automatic generation (top down) [Foley] By demonstration [Myers]
Other categories Constraint-based [Hudson] User-oriented [Moran]
5-21
Example: Visual Programming for UI
Layout vs. Behavior UIMS, with state diagrams for behavior
[Jacob, “A state transition diagram language...”]
Visual Basic, pre-programmed behavior
5-22
Example: State Diagram as UIDL
Command-driven Graphical Report Editor
[Jacob, “Using formal specifications...”]
5-23
UI Evaluation
Directly from UIDL Evaluate interface without building system Emerging area, but lack underlying formal knowledge
of UI design Analyze user interface to detect specific properties
[Darlington et al.] Apply human performance models [Card] Automated screen design critique [Tullis] Prove safety-critical properties
Screen mockup, prototyping tools “Slide show" of canned screens based on user inputs UIMS sans application (use stubs for semantics) Visual Basic, Powerpoint, plain HTML also used this
way
5-24
UI Tools: Standardization Early:
Many desktop platforms No standardization
Current: ~3 major platforms, standard widgets Settled into stable toolkits, widget libraries, GUI class
libraries UIMS seems like overkill
Emerging: Mobile, automobile Diverse platforms, no standardization Wider range of interaction techniques Model-based development, UIMS redux
Future: Same issue for new non-WIMP interaction styles Still wider range of possible interactions
5-25
UI Tools: Trends
UI toolkits, libraries Become important, integral part of software
development tool sets e.g., Macintosh Cocoa UI features are well integrated
Good uptake despite unusual programming language
Web browser as UI platform Cloud computing Chrome-only OS New tools for web UIs, but still clumsy, poorly
integrated Making cross platform toolkits less significant
5-26
UI Tools: Mobile Devices Basic HCI development and design principles
Largely GUI, not new interaction style + gestures, GPS, ... Use context: Full attention or not?
Development tools Early: Clumsy tools, restricted UI designs
Much like early GUIs, function key-driven (pre-mouse) GUI Current: UIs and tools very similar to standard GUI
Current Future Single thread I/O Parallel, asynchronous
dialogues; may be interrelated
Discrete tokens Continuous inputs and responses (plus discrete)
Precise tokens Probabilistic input, not easily tokenized
Sequence, not time Real-time requirements, deadline-based
Explicit user commands
Passive monitoring of user
5-29
Reality-Based Interaction Connects several emerging interaction styles
Understand together as a new generation of HCI through RBI [Jacob et al., “Reality-Based Interaction...”]
Computer interaction more like rest of world Exploit skills and expectations user already has about
simple, everyday, non-digital world Reality + extensions
Design tradeoffs:
5-30
Tangible User Interfaces
Simple, transparent mechanical structures
Use knowledge of physical world to operate
Augment physical objects with digital meaning
Combine physical + digital representations
Exploit advantages of each
[Jacob et al., “A Tangible Interface...”]
[Zigelbaum et al., The Tangible Video Editor...”]
5-31
Technology for TUI
Sensors Produce signal in response to change in surroundings
Actuators Produce physical change in response to signal
Microcontroller Communicates with devices and with main computer
Other technologies RFID Computer vision ...
Wider range of I/O events than GUI Development methodology, tools
None!
5-32
A Visual Language for Modeling Tangible User Interfaces [Shaer]
TAC paradigm Each TUI consists of token
within a constraint Same object may
sometimes be token, sometimes constraint
Two tier model fits well Dialogue (states,
storyboard) Interaction (especially
continuous)
TUIML
5-33
Conclusions: 1. Research Background
Background for following current research in user interface software
Interaction styles Dialogue independence Four levels of design User interface management system (UIMS) User interface description language (UIDL) Goal: Generate UI from abstract description Types of software tools New interaction styles (non-WIMP) and requirements Easier to use, harder to program?
5-34
Conclusions: 2. Practice
What research ideas can be applied in building user interfaces today?
Separate interface from application programming Model-View-Controller architecture Use specialized tools (UIMS) for leverage Use specialized languages (UIDL) for perspicuity and
ReferencesW. Buxton, M.R. Lamb, D. Sherman, and K.C. Smith, ''Towards a Comprehensive User Interface Management
System,'' Computer Graphics 17(3) pp. 35-42 (1983).
S.K. Card, T.P. Moran, and A. Newell, ''The Keystroke-Level Model for User Performance Time with Interactive Systems,'' Comm. ACM 23 pp. 396-410 (1980).
J. Darlington, W. Dzida, and S. Herda, ''The Role of Excursions in Interactive Systems,'' International Journal of Man-Machine Studies 18 pp. 101-112 (1983).
J. Foley, W.C. Kim, S. Kovacevic, and K. Murray, ''Defining Interfaces at a High Level of Abstraction,'' IEEE Software 6(1) pp. 25-32 (January 1989).
J.D. Foley, A. van Dam, S.K. Feiner, and J.F. Hughes, Computer Graphics: Principles and Practice, Addison-Wesley, Reading, Mass. (1990).
J.D. Foley and V.L. Wallace, ''The Art of Graphic Man-Machine Conversation,'' Proceedings of the IEEE 62(4) pp. 462-471 (1974).
H.R. Hartson and D. Hix, ''Human-computer Interface Development: Concepts and Systems for its Management,'' Computing Surveys 21(1) pp. 5-92 (1989).
S.E. Hudson, ''Graphical Specification of Flexible User Interface Displays,'' Proc. ACM CHI'89 Human Factors in Computing Systems Conference pp. 105-114 (1989).
R.J.K. Jacob, "A Specification Language for Direct Manipulation User Interfaces," ACM Transactions on Graphics, Vol. 5(4) pp. 283-317 (1986) http://www.cs.tufts.edu/~jacob/papers/tog.pdf].
R.J.K. Jacob, ''A State Transition Diagram Language for Visual Programming,'' IEEE Computer 18(8) pp. 51-59 (1985) [http://www.cs.tufts.edu/~jacob/papers/ieeecomputer.pdf].
R.J.K. Jacob, "Using Formal Specifications in the Design of a Human-Computer Interface," Communications of the ACM, Vol. 26(4) pp. 259-264 (1983) [http://www.cs.tufts.edu/~jacob/papers/cacm.pdf].
R.J.K. Jacob, A. Girouard, L.M. Hirshfield, M.S. Horn, O. Shaer, E.T. Solovey, and J. Zigelbaum, "Reality-Based Interaction: A Framework for Post-WIMP Interfaces," Proc. ACM CHI 2008 Human Factors in Computing Systems Conference pp. 201-210, ACM Press (2008) [http://www.cs.tufts.edu/~jacob/papers/chi08.pdf].
R.J.K. Jacob, H. Ishii, G. Pangaro, and J. Patten, "A Tangible Interface for Organizing Information Using a Grid," Proc. ACM CHI 2002 Human Factors in Computing Systems Conference pp. 339-346, ACM Press (2002) [http://www.cs.tufts.edu/~jacob/papers/chi02.pdf].
R.J.K. Jacob, Q. Limbourg, and J. Vanderdonckt (Eds.), Computer- Aided Design of User Interfaces IV, Kluwer Academic Publishers, Dordrecht (2005).
J. Johnson and A. Henderson, "Conceptual Models: Begin by Designing What to Design," interactions, Vol. 9(1) pp. 25-32 (January 2002).
D.J. Kasik, ''A User Interface Management System,'' Computer Graphics 16(3) pp. 99-106 (1982).
T.P. Moran, ''The Command Language Grammar: A Representation for the User Interface of Interactive Computer Systems,'' International Journal of Man-Machine Studies 15 pp. 3-50 (1981).
B.A. Myers, ''User Interface Software Tools,'' ACM Transactions on Computer-Human Interaction 2(1) pp. 64-103 (March 1995).
B.A. Myers, Creating User Interfaces by Demonstration, Academic Press, Boston (1988).
B. Myers, S.E. Hudson, and R. Pausch, "Past, Present, and Future of User Interface Software Tools," ACM Transactions on Computer- Human Interaction, Vol. 7(1) pp. 3-28 (March 2000). Also reprinted in Human-Computer Interaction in the New Millenium, ed. J.M. Carroll, Addison-Wesley/ACM Press, Reading, Mass., 2002, pp. 213-233.
W.M. Newman, ''A System for Interactive Graphical Programming,'' Proc. Spring Joint Computer Conference pp. 47-54, AFIPS (1968).
D.R. Olsen, Building Interactive Systems: Principles for Human-Computer Interaction, Course Technology Inc. (2009).
D.R. Olsen, User Interface Management Systems: Models and Algorithms, Morgan Kaufmann, San Mateo, Calif. (1992).
F. Paterno, Model-Based Design and Evaluation of Interactive Applications,, Springer-Verlag, London (2000).
O. Shaer and R.J.K. Jacob, "A Specification Paradigm for the Design and Implementation of Tangible User Interfaces," ACM Transactions on Computer-Human Interaction (2009) [http://www.cs.tufts.edu/~jacob/papers/tochi.shaer.pdf].
B. Shneiderman, C. Plaisant, M. Cohen, and S. Jacobs, Designing the User Interface: Strategies for Effective Human-Computer Interaction, Addison-Wesley, Reading, Mass. (2009).
E.R. Tufte, ''Visual Design of the User Interface,'' IBM Corporation, Armonk, N.Y. (1989).
5-37
T.S. Tullis, ''A Computer-Based Tool for Evaluating Alphanumeric Displays,'' Proc. INTERACT '84 Conference, B. Shackel, ed. (1984).
J. Zigelbaum, M. Horn, O. Shaer, and R.J.K. Jacob, "The Tangible Video Editor: Collaborative Video Editing with Active Tokens," Proc. TEI 2007 First International Conference on Tangible and Embedded Interaction pp. 43-46 (2007) [http://www.cs.tufts.edu/~jacob/papers/tei07.zigelbaum.pdf].