DESIGN OF INTERACTIVE SYSTEMS - web.njit.eduturoff/coursenotes/IS732/newvg173…  · Web viewRorschach Blot. PSYCHOLOGICAL ROLES (Rorschach Blots) OF COMPUTERS. ... Speaker’s economy

DESIGN OF INTERACTIVE SYSTEMSOverheads Set I

by Murray TuroffDepartment of Computer and Information ScienceNew Jersey Institute of TechnologyNewark NJ, 07102TEL: 973 596 3399email: [email protected] homepage://eies.njit.edu/~turoff/© Copyright 1998 Murray Turoff

CONTENTS•1Subject One: Fundamental Concepts, Observations and

Problems•2Subject Two: Dimensions of Interface Design•3Subject Three: The Design Process•4Subject Four: Protocol Analysis•5Subject Five: Indexing & Searching

FUNDAMENTAL CONCEPTSSubject One© copyright 1998 Murray Turoff

Where is the wisdom we lost in knowledge?Where is the knowledge we lost in information? T. S. Eliot (The Rock)

ATMOSPHEREHUMAN COMPUTER•1Slow Rapid•2Sloppy Rigorous•3Forgetful Precise•4Implicit Explicit•5Subjective Objective•6Brilliant StupidHow do we design a communication process between the two?

1

WHY INTERACTIVE I•1Iterative Problem Solving•2Unpredictable Sequences•3Tool Flexibility•4Impossible Manually

•5Creativity and Quality•6Collaboration, Coordination & Communications

WHY INTERACTIVE II•1Enhancement (save time and/or effort)

•2Speed, quantity, memory•3Enjoyment & Challenge•4Sublimating & Escape•5Power & Control•6Solving the Wrong Problem

INFORMATION DOMAINS OF USERS I•1Single Function Tasks

•2simple inquiry / calculations / messaging•3Structuring Tasks

•4Organizing / filtering / summarizing•5Status Briefing / Report Generation•6Tracking / Monitoring Tasks•7Creation / Composition / Modeling

INFORMATION DOMAINS OF USERS II•1Challenges for Individual Support

•2Exception Reporting•3Diagnosis•4Discovery•5Hypothesis Testing and Analysis

•6Challenges for Group Support•7Planning, Decision Analysis and Resource Allocation

•8Command, Control & Coordination

2

STAGES OF USER EVOLUTION•1Uncertainty Phase•2Insight Phase•3Incorporation Phase•4Saturation Phase

USER CHARACTERISTICS•1High Cognitive Variability, Problem Solvers, Creative,

Demanding•2Novice, Casual, Intermediaries, Operators, Routine•3Experienced, Frequent, Power•4Children to Adults•5Clerk, Analyst, Engineer, Scientist, Manager, Executive•6Results:

•7Different User Roles in one System•8Multiple Interface Methods Required

DESIGN METHODS I•1Comparison/Differentiating •2Designing / Requirements Analysis

•3Task / Macro•4Cognitive / Micro•5Group / Micro & Macro•6Process Design / Integration•7Enterprise Design/Organizational•8Technology / Environmental

DESIGN METHODS II•1Enhancements / Evolution•2Visioning / Normative Planning

•3Social Engineering•4Goal Analysis•5Enterprise Design

3

THE DEVELOPMENT PROCESS

Mental Model

Interface Metaphor

ImplementationModel

FunctionalOpacity

SystemOpacity

Real World

RequirementsModel

System Analysis

Experience

Testing

Training

Usage

Influence

InfluenceVirtualExperience

Validation

Evaluation

SystemDesign

MODELS I •1Mental Models of users resulting from their “real” world experiences

•2Requirements Model determined by the system analysis process acting on user mental models.

•3Implementation Model is the actual internal design model of the supporting software processes.

MODELS II•1Real World is the actual system being model.•2Interface Model/Metaphors is the mediator between the users mental model and the systems implementation model.

PROCESSES I•1Comparison of the Implementation model to the

•2Requirements model is TESTING•3Mental Model is EVALUATION•4Real World is VALIDATION•5Interface Model is TRAINING/LEARNING

4

PROCESSES II•1Functional Opacity is created by large differences

between the Interface Model and the Mental Model•2System Opacity is created by large differences between

the Interface Model and the Implementation Model.•3Extreme functional opacity or system opacity can lead

to cognitive dissonance.

VIRTUALITYVirtuality is the process whereby the implementation model in

the computer replaces the real world model in a system made up in part of people.•1Information Systems are usually design on a descriptive

basis; however, they often become prescriptive.•2Designers need to anticipate prescriptive impacts of their

systems.•3The design of an interactive system can be the design of a

social system

GENERAL SYSTEMS VIEW•1Interactive Systems are open, dynamic, and adaptive in

the long run•2They will be changed by the organizational/social

environment and they will influence changes in those environments.

Law of Requisite Variety Applies: The inputs to a system must be as rich/variable as the outputs if one wishes to control the outputs of a system.

FUNDAMENTAL PROBLEM I•1Proper Level of Tools / Objects / Data Structure

•2Too Micro / Low Level•3Difficult to work with•4More effort and time

•5Too Macro / High Level•6Inflexible•7Difficult to Change

5

FUNDAMENTAL PROBLEM II•1Gulf of Execution

•2Translating goals, intentions to actions•3Functional Opacity

•4Gulf of Evaluation•5Translating output display to evaluation•6Not System Opacity but related•7Excessive trial and error

PLAYING MUSIC PROBLEM•1What are the principle alternative ways to make music?•2Designing a system any idiot can use and only idiots

will use it!•3Novice Friendly, Experienced Hostile•4Two Separate Dimensions

•5Beginning users / Experienced users•6Non Domain Experts / Domain Experts

COMMON PSYCHOLOGICAL EFFECTS•1Fishbowl•2Bully•3Peephole•4Concrete•5Clutter (Confusion)•6People Angst•7Computer Angst•8Rorschach Blot

PSYCHOLOGICAL ROLES (Rorschach Blots) OF COMPUTERS

Evaluator, Magician, Helper, Entertainer, Companion, Challenger, Foe, Mentor, Accomplice, Producer, Overseer, Dictator, Priest, Servant, Picky Parent, Unruly Child, God, etc.

•1Computers can be made to appear to be emotional and/or intelligent.

•2Providing computer literacy almost an ethical issue

6

USER RESPONSE TO INADEQUATE SYSTEM•1DIS-USE: Turn to other sources•2MIS-USE: Using inappropriate ways•3PARTIAL USE: Use of wrong subset•4DISTANT USE: Use of intermediary•5MINIMUM USE: Recipe (IRS type) interaction•6NON-USE: Avoiding the system•7MODIFICATION OF TASK: Change the task to fit the system

(solving wrong problem)•8COMPENSATORY ACTIVITY: User has to do more•9DIRECT PROGRAMMING: User modifies system/System encourages

unique interfaces

Sources of Knowledge and Wisdom I•1General Social Sciences Methods

•2Controlled Experiments•3Focus Groups•4Interview Methods•5Participant Observation•6Survey Methods•7Longitudinal studies•8Field Studies

Sources of Knowledge and Wisdom II•1Industrial Engineering

•2Human Factor studies•3Physiological response studies•4"time and motion" studies (Method)•5Human Monitoring (Method)

•6Psychology•7Human cognitive processes•8Human problem solving•9Protocol Analysis (Method)•10Human psychology and mental ability measurement

7

Sources of Knowledge and Wisdom III•1Management Sciences

•2Cost/Benefit Analysis (Method)•3Efficiency, Effectiveness, Quality, Opportunity, Regret,

Productivity•4Delphi (Method)•5Perceptions and System Monitoring

•6Measures of user satisfaction, efficiency of use, quality and effectiveness of applications

•7Management/Operational practices•8Performance of Systems/Technology

Sources of Knowledge and Wisdom IV•1Sociology

•2Conceptions of technology•3Social and organizational systems•4Group processes and behavior•5Environmental interaction

•6Information Science•7Data base Technology•8Cognitive use of data and information •9Information Retrieval

Sources of Knowledge and Wisdom V•1Anthropology

•2Understanding metaphors and culture (Method)•3Cultural adaptation•4Human learning process

Sources of Knowledge and Wisdom VI•1Archaeology

•2"study of artifacts" (Method)•3Tools and toolmaking

•4Philosophy•5"virtual reality"•6Design of Social Systems•7Virtuality

8

Sources of Knowledge and Wisdom VII•1Computer Science & Information Systems

•2Designer Wisdom•3Expert Systems•4Interface Design•5Applications•6Computer Mediated Communications•7Systems Analysis & Requirements Development•8Process Reengineering•9Enterprise Design

DIMENSIONS OF INTERFACE DESIGNSubject two© copyright 1998 Murray Turoff

CRITERIA FOR FACTOR DIMENSIONS•1Can it be perceived?•2Can it be measured?

•3Reproducible (repeatable)•4Reliable (valid measure)

•5Orthogonal (Fundamental/Unique)•6Can it guide design?

•7Relate to Interface design choices•8Can it evaluate design?

•9Used to compare different designs•10Can it sensitize designers?

•11Make them better designers

BASIC CATEGORIES OF DIMENSIONS•1Foundation Factors•2Understanding & Ease of Learning•3User Sense of Control•4Effectiveness•5Psychological & Sociological•6Administrative & Management

9

FOUNDATION FACTORS•1Timeliness (Objective) •2Responsiveness (Cognitive)•3Reliability•4Accessibility / Convince•5Efficiency / Least Effort•6Security•7Accuracy•8Protection/Bullet-proofing

UNDERSTANDING / EASE OF LEARNING I•1Guidance / Informativeness•2Conciseness / Brevity•3Clarity / Simplicity•4Comprehension•5Segmentation / Decomposition

UNDERSTANDING / EASE OF LEARNING II•1Consistency•2Retention•3Specificity•4Familiarity•5Scaffoling

SENSE OF CONTROL I•1Leverage / Modifiability•2Manipulability•3Closure / Confirmation / Notification•4Feedback / Sense of causality (Synchronous)•5Contextual Visibility•6Multi-tasking•7Process Control / escape / interrupt / restart

10

SENSE OF CONTROL II•1Forgiveness•2Transparency•3Flexibility / Cognitive Adaptation•4Predictability / Regularity•5Tracking / Monitoring

•6Backtracking / Auditing / History•7Forecasting / Anticipating / Alerting•8Backup / Undo / Listing

EFFECTIVENESS I•1Task Functionality

•2Generality•3Matching•4Completeness•5Abstraction / Leverage •6Adaptability / Manipulability

EFFECTIVENESS II•1Integration / Connectivity•2Resiliency / Robustness / Richness•3Relevance of objects and functions•4Precision (minimize non relevant material)

PSYCHOLOGICAL & SOCIOLOGICAL I•1Ethical / Honest / Private•2Aesthetic / Pleasing / Artful•3Interesting / Challenging / Fun•4Self Image Enhancement•5Peer Relations / Status•6Sense of Community•7Humanization / Polite

11

PSYCHOLOGICAL & SOCIOLOGICAL II•1Satisfaction

•2Self, System, Group / Organization, Task / Job•3Motivation•4Expectations / “Self full filling prophecy”•5Perceived Utility•6Change & Social Inertia•7Feeling of Participation

ADMINISTRATIVE•1Training / Documentation•2Maintenance of System •3Job Enhancement•4Human Help & Contact•5Organization Relationships•6System Evolution / Modifiability•7Evaluation / Feedback •8Costs and Charging•9History & Confidence

EXAMPLES OF CONFLICTS AND TRADEOFFS•1Comprehension <> Segmentation•2Consistency <> Efficiency•3Consistency <> Least Effort•4Conciseness <> Informative•5Conciseness <> Closure•6Resiliency <> Ease of Learning•7Task Generality <> Task Matching•8Specificity <> Familiarity•9Leverage <> Manipulability

COMMON DESIGN FAULTS I•1Poor Interactive Sequences•2Longer to do than manual (e.g. reading)•3No tolerance or respect for human error•4No flexible parsing (rigid syntax)•5Wrong functionality•6Start-stop hassle / Interruption handling

12

COMMON DESIGN FAULTS II•1Integration across functions•2Poor documentation and help•3Inconsistent Metaphors•4Lack of flexibility•5Applicability to broad range of tasks

COMMON DESIGN FAULTS III•1Multiple approaches to a given task•2Multiple ways of invoking tasks•3Alternative sequencing of functions•4Adapt to different user styles / user types

INTERACTION METHODS I•1MENUS: comprehension•2COMMANDS: flexibility•3LISTS: leverage •4FORMS: clarity•5DIALOGUE: guidance•6WINDOWS: multi-tasking•7ICONS: object representation•8GUI: special representations•9DIRECT MANIPULATION: control

INTERACTION METHODS II•1MACROS: leverage•2MIMICKING / RECORDING: task matching•3ANIMATION: event sequencing•4MODELS: abstraction•5LANGUAGES: semantic representations•6SCRIPTING: leverage, task matching•7VIRTUAL REALITIES: understanding•8AI AND EXPERT SYSTEMS: efficiency

13

THE DESIGN PROCESSSubject Three© copyright 1998 Murray Turoff

DOING DESIGN•1Any type of design component can be the stimulus for an idea

•2Design can occur in any order through the maze of component relationships

•3Any incident of a component can be the stimulus for the idea process

•4Design is a non linear thought process

DESIGN COMPONENTS I•1Goals and Objectives•2User Tasks & Requirements•3System Metaphors

•4System Artifacts/toolsISSUES

•1How to make the computer recipe system more valuable to the user than any of the regular physical systems he or she can use?

•2How to extend the concept of what is in a recipe to be more than what the user initially conceives it to be?

RECIPE SYSTEM OBJECTIVES I•1The user should be able to handle thousands of recipes.

•2Provide ways of organizing the information that is superior to that provided by books and index card files.

•3Identify useful functionality that is easier to take advantage of on the computer.

RECIPE SYSTEM OBJECTIVES IIInsure that individuals can learn to use this system very easily

and be able to accomplish frequently occurring operations

14

with a minimum of effort.•1Provide facilities for the user to be able to learn more about cooking.

USER TASK: search requirements•1Meals and meal parts (e.g. breakfast, snack, feast, main

course, soup, desert, drinks, etc.)•2Ingredients (meat, eggs, fruit, vegetables, etc.)•3Nutritional (low fat, vegetarian, high protein, low salt,

etc.)•4Cultural (Chinese, various holidays, religious, ethnic,

national, etc.)•5Taste (hot, sour, sweet, bitter, coarse, fine, aromatic,

fragrant, etc.)

RECIPE SYSTEM METAPHORS•1Recipe, Meal•2Cookbook•3Card File•4Scrap Book •5Picture Metaphor

•6Dining Room, Kitchen, Root Cellar, Pantry, Menu (soup plate, etc.)

RECIPE TOOLS/ARTIFACTS•1Nutritional analysis

•2e.g. calorie counter•3Shopping list accumulation•4User / Family Preferences Profile

•5e.g. Filters: likes, dislikes, necessary (low fat, diabetic, low salt)

•6Glossary, Historical Information•7Food and meal layout function

TOOL EXAMPLE: RDA calibrationFunction to modify Recommended Daily Allowance (RDA)

values for individual (lower salt, etc.) which would change nutritional calculation

Total Fat, Saturated fat, Polyunsaturated fat, Monounsaturated fat, Cholesterol, Sodium, Carbohydrates, Dietary Fiber, Protein, Sugars, Vitamins, etc.

•1Would require a master ingredient list giving all such

15

information per unit of ingredient.

16

DESIGN COMPONENTS II•1Objects / sub-objects

•2Object Parts•3Representations

•4Shortest: used to shorten lists•5Abstract: provide overview•6Content: to fill in total object

RECIPE OBJECT PARTS•1"title"•2Descriptors for indexing•3Auxiliary information: e.g. number of servings, side

dishes, drinks•4List of "ingredients" and the "amount" of each•5Set of cooking "instructions"•6User notes, markups•7Modification alternatives for special occasions, added

by user

RECIPE REPRESENTATIONS•1Shortest: Unique title•2Abstract: title, index terms, auxiliary data•3Content: ingredients, instructions, links

OTHER POTENTIAL OBJECTS•1Meals•2Key word index•3Ingredients index•4Marked list

MEAL/MENU Object•1Classical Meal Object

Appetizers, Drinks, Breads, Soup, Salad, Cleanser, Main course(s), Vegetables, Starch, Desert, After dinner drinks

•1User should be able to define meal structure for different types of meals dinner, lunch, breakfast, brunch, family style, picnic, etc.

17

•1Compilation of all ingredients for shopping list

18

DESIGN COMPONENTS III•1Functions on objects

•2Generic (detail depends on object)•3Explicit (self defined)•4Strategic choice sets•5Reactive choice sets•6Controls

GENERIC & EXPLICIT FUNCTIONS•1Generic:

Get, Find, Create, Modify, Delete•1Explicit Commands for Index

Add, Remove (term), Index (as verb) object, Change (term), Merge (terms), Split (term), Order (index), vote

STRATEGIC MENU/CHOICES

Object Number Actions

Recipes 3,455 Get

Meals 238 Find

Lists Changes

Keys 485 Create / Add

Ingredients 9,989 Modify /Change

Marked 35 Delete / Remove

19

EXAMPLE REACTIVE FUNCTIONS•1Acting on some subpart of an object

•2Modify/add notes •3Nutritional Analysis•4Find Substitute/Alternative•5Add to shopping list

REACTIVE MENU SAMPLE•1Scan/List/View•2Find•3Mark/unmark•4Add (index term/instruction/ingredient))•5Remove (index term)•6Controls: Home (escape), End (finish), Help, Show

EXAMPLES OF CONTROLS•1Go (how far), back, forward, elsewhere, open/close/size

windows, etc.•2Print/File (how much): object, screen, page, list, window,

link, etc.•3Task control, confirm, quit, help, undo, escape, finish,

interrupt, continue, copy, cut, paste, open, close, etc.One can trace the evolution of interface design through the

evolution as to what has become a commonly acceptable control “tool”

DESIGN COMPONENTS IV•1Modifiers and Status states

•2Subsets, tracking•3creates meaningful lists

•4Lateral semantic linkages•5Shared processes

•6List Processing•7Searching

20

MODIFIERS•1Create subset/list of objects•2Examples

•3Authored recipes•4Incomplete recipes•5Sent messages•6New messages

LINKAGES I•1Meal to recipes•2Recipe to meals•3Ingredient

•4as recipe•5to details•6recipes/meals utilizing it•7to substitutions

•8Number of servings to change program

LINKAGES II•1To added Information

•2cooking method•3utensils•4nutritional details

•5Name to similar recipes•6Related items: e.g. wine, side dishes•7Time to prepare, to cooking

LINKAGE TYPES: Open Ended•1Two way usually desirable

•2Part of (recipe in meal)•3List generation (all meals with recipe)

•4More than one link from object/anchor•5Alternative/substitution (of ingredient)•6Explanation/Information (about ingredient)

•7Association/equality (other recipe)•8Trigger program (e.g. servings, nutrition)

21

OBSERVATION ABOUT RECIPE•1Natural contextual menu

•2user knows context•3user can be taught context

•4User can learn easily through trial and error•5Ease of learning clear

LIST PROCESSING•1Adding and removing: index terms, indexed objects•2Dividing, Merging, or Sorting lists•3Comparing lists (multiple object entries)•4Inverting (all occurrences)•5Expanding, Contracting levels•6Marking

KEY LIST SAMPLE I•1meat

•2veal•3...

•4beef•5sweetbreads•6brain•7red meat

•8roasts•9 ...

KEY LIST SAMPLE II•1roasts

•2London Broil•3Eye•4Rump•5...

•6steak•7...

•8processed•9hot dogs

22

ALTERNATIVE KEY STRUCTURES•1Turkey

•2Holidays•3Thanksgiving•4Christmas

•5Leftovers•6...•7Fowl

•8Turkey•9White meat

•10Breast

KEY LIST OPTIONS•1Meat levels: 9 Items: 455•2Use + or ... or "number" to expand or contract

•3Allow objects in multiple locations•4meat.beef.red_meat.steaks.skirt (Shows where it came from)

SEARCH ISSUES I•1What specific reactive searches are needed throughout the interface

•2Initiating the search and being informed of the results with respect to the amount of material found.

SEARCH ISSUES II•1Deciding and indicating what is to be retained or discarded in a search result.

Whether to initiate a search of material found or to search the material not retrieved and how to combine this with the prior searches.

DESIGN COMPONENTS V•1Object Formats•2Screen layouts

•3Workspace, status areas•4Control areas, message areas

23

OBJECT FORMATRattlesnake Stew Created: 8/12/91 Modified: 6/5/92

Keys: /meat.snake/exotic/rich/ American.southwest.Indian/Servings: 2 Preparation Time: 90 min. Last used: 12/8/92 Times used: 15Characteristics: High Fat, StewedUtensils: Simmering PotRemarks: Supplied by John Franklin from my National Guard

unit. Tastes very much like rich oily chicken.

INTEGRATION EXTENSIONS•1Electronic Shopping•2Maintaining inventory•3Exchange recipes with other users•4Group Meal Planning•5Education: Virtual World Recipe System

•6Being able to actually replicate the cooking process in the virtual world.

•7Being able to have an instructor join you in that environment.

•8Being able to cook as a group.

DESIGN COMPONENTS VI•1Processes and Closure

•2Processes and events that take place independent of user.

•3Interaction process & modes•4The states and modes a user can be in.

•5User Object lists•6The lists of objects the user will need or desire to manipulate.

DESIGN COMPONENTS VII•1Error conditions

•2Identification of possible errors that the user could make.

•3System Messages•4Notifications, Closures, Confirmations, and Error Messages.

•5Necessary Help•6Identification of what is unclear or not self evident in

24

the interface.

25

DESIGN COMPONENTS VIII•1For Specialized Systems

•2CMC: roles, notifications•3DATABASES: forms, entity models, reports•4COMMUNICATIONS: protocol scripts/templates•5HYPERTEXT: anchor, linkage types

Design Levels IGENERAL SPECIFIC DETAIL

Goals User TaskMetaphorsObjects Heading, Abstract

ContentObject partsSub-objectsFormatsLinkages

Functions / Actions GenericLinks

Explicit, Controls

Design Levels 2GENERAL SPECIFIC DETAIL

Strategic Choice Sets Reactive Choice SetsShared Processes Searching

List ProcessingInteraction States Screen layouts

Error ConditionsNecessary Help

Workspace, Status, Control, & System message area

26

PROTOCOL ANALYSISSubject Four© copyright 1998 Murray Turoff

Language is the dress of thought Samuel Johnson

WHY DO IT?•1Objective: To discover the process a person goes through in solving a problem.

•2Uses: Learning Cognitive Processes, Developing Expert System material, evaluating interfaces.

KEY ASSUMPTION•1Cognitive processes that generate verbalization are subset of those that generate behavior

•2Example: Lisa Learning, by Carroll and Mazur, IEEE Computer November 1986.

CONCERNS ABOUT THE METHOD•1Subjects may have incomplete knowledge of their

thinking processes; therefore the record may be incomplete.

•2Subjects may not have an accurate understanding of the processes of which they are aware.

•3The thinking process may be distorted by the thinking out loud process.

•4Ambiguity in language may lead to miscommunications.

NECESSARY ATMOSPHERE•1Honesty•2No evaluation of subject•3No pressure for performance•4No introduction of bias•5No contamination of mental process (e.g.

27

help)•6Reciprocity and Respect

28

STIMULUS RESPONSE METHOD•1Different stimulus may produce different mental

behaviors•2Do you know the capital of Sweden?•3Which of the three: Oslo, Stockholm, or Copenhagen is the capital?

•4Name the capital of Sweden.•5How much does it cost to fly and elephant from

Thailand to Los Angeles?•6How long is the Nile river?

MODES OF PROBING•1Talk Aloud, Think Aloud: While information is attended.

•2Concurrent Probing: While in short term memory.

•3Retrospective Probing: After completion of the task.

PROTOCOL ANALYSIS CONDITIONS•1Subject asked to verbalize what they are thinking•2Subject is not being evaluated•3Observer must not participate in process•4Observer must not aid the subject•5Subject providing knowledge of how they solve or a problem (or learn a system)

PROTOCOL ANALYSIS PROCESS I•1Present the subject a written explanation of what is taking place.

•2Explain that this is to evaluate the system and not them

•3Explain you are there only to observe and can not help them.

•4Present in writing a task written the user terminology and the way the user would think about it.

29

PROTOCOL ANALYSIS PROCESS II•1Total time should take between thirty minutes to an hour if no major problem encountered.

•2Observe and record (video, voice tape, PC interactions, notes and coding)

•3Only interrupt user for further verbalization if it is unclear what they are doing.

PROTOCOL ANALYSIS PROCESS III•1Ask the user to describe what he or she is doing outloud.

•2Ask the user to go through all the terms on the screen and explain what they think they mean

•3Ask the user to forecast what they think an action will do

•4Carry out the task on the system

PROTOCOL ANALYSIS PROCESS IV•1Give help only if user is at a dead end•2Questions you can ask during if necessary

•3Why do you do/say that?•4What is troubling you?•5How do you know that ...?•6Why do you do it this way?

•7Save retrospective questions for end of session.

MAJOR LIMITATION•1Can not use it on a task that the user is not familiar with.

•2Requires training on the task first.TYPICAL WRITTEN INSTRUCTIONS

•1PLEASE EXPLAIN:•2What you think something means.•3What you are trying to do.•4What confusion or concerns you have.•5What you expect to happen next.•6What you don’t know the meaning of.

30

CODING SCHEMA FOR VERBALIZATION I•1EXAMPLE I:

•2Intentions: goals, shall, will, must, have to•3Cognitions: current attention situation, define, understand

•4Planning: If x than y•5Evaluation: Yes, No, Damit, Fine•6Changing conditions \ view of the problem

CODING SCHEMA FOR VERBALIZATION II•1EXAMPLE II:

•2Surveying given information•3Generating new information•4Developing a hypothesis•5Unsuccessful solutions•6Self reference or criticism•7Silence

PROTOCOL ANALYSIS OBSERVATIONS I•1Verbalization occurs only 30% to 50% of the time.•2Subjects cannot verbalize when:

•3Reading text•4Doing intense cognitive activity•5Making choices

•6Subjects have to slow down to verbalize•7Subjects will tend to be more careful as a result of

verbalization and observation

PROTOCOL ANALYSIS OBSERVATIONS II•1Experts on a problem verbalize a lot more than non experts (double)

•2Experts have more difficulty verbalizing at a very detailed level with respect to the problem with which they are dealing with.

•3Experts usually spend more time in planning and in the restructuring of the problem.

31

INCREASING VERBALIZATION•11. Hold back stimulus or encourage slowness.•22. Segment stimulus (subtasks)•33. Interrupt with pre-arranged signal or set point to pause•4e.g. when you are ready to indicate an action, first explain what you think everything means on the screen.

OBJECTIVES FOR INTERACTIVE SYSTEMS I•1Determine their understanding of terms in the interface

•2Understand the cause of errors or misinterpretations

•3Determining missing functionality or user requirements

OBJECTIVES FOR INTERACTIVE SYSTEMS II•1Determining reactions to and utility of features

•2Determine the utility of a metaphor for learning

•3Determining the utility of help and guidancePROTOCOL ANALYSIS ADVANTAGES

•1A lot less effort than other approaches.•2Can be done with prototype or mockup before any coding.

•3Learning how user approaches task•4Finding major mistakes in design •5Can learn attitude•6Rapid feedback from small samples•7Also useful for understanding user task

PROTOCOL ANALYSIS REQUIREMENTS•1Subjects must be representative•2Three subjects for each distinctive type of

32

user and for each different set of tasks•3Instructions simple•4Observe only

PROTOCOL ANALYSIS QUESTIONS•1Can ask/say:

•2Please explain your choice.•3What are you thinking?•4What does that term mean?

•5Should not ask:•6Why did you do that?•7What does “append” do?

•8Have physical signal for interrupt

PROTOCOL ANALYSIS: HOW TO I•1A one page explanation to the subject•2Set of written tasks in user terms•3Subject should only spend 40-60 minutes.•4Categorization scheme for recording•5Tape record their verbalizations

PROTOCOL ANALYSIS HOW TO II•1Retrospective questionnaire for end•2Retention of major concepts•3Perceived utility of features/functionality•4Do not try to test everything •5At least three subjects on same task•6Be specific about user explaining choice they are about to make

UTILITY OF MOCK UP•1Ease of understanding (current knowledge)•2Ambiguity in terms•3Confusion generation•4Loss of continuity•5Developing on line help•6Marketing and acceptance

33

UTILITY OF WORKING SYSTEM•1Ease of learning (new knowledge)•2Error impact & Utility of feedback•3Ease of exploration•4Realistic task execution•5Developing final user documentation•6Utility of new features (beyond current system)

Wizard of Oz System•1Allows simulation of working system•2Dummy output data stored•3Working choices for moving among screens and windows

•4More software to do this appearing on the market

BASIC QUESTIONS I•1Do the terms used on the screen mean to the subject what the designer thought they would mean?

•2Do the alternatives presented at that point in the interaction include what the subject wishes to do?

•3Is the help material or the system messages understandable or relevant to the needs of the user?

BASIC QUESTIONS II•1Does the subject have difficulty locating or perceiving things on the particular screen?

•2Does the subject utilize the sequences of operations that the designer expected to be utilized in accomplishing a given task?

•3Can the user utilize the interface metaphor for learning the system?

•4What type of errors is the user making and why?

34

WHEN TO USE•1Protocol analysis should be used•2Before system requirements are finalized•3After every major change to requirements or interface

•4Before instillation of system for users•5Before introduction to new user population type

INDEXING & SEARCHINGSubject Five© copyright 1998 Murray Turoff

"When I use a word," Humpty-Dumpty said, "it means just what I choose it to mean -- neither more nor less." --Lewis Carroll, Alice in Wonderland

PART OF THE PROBLEM Bible, Holy Bible, La Sainte Bible, Biblis Sacra, Bible wordo,

biblia, Scriptures, Holy Scriptures, The Scriptures, Sacra Scrittur, Saintes, New Testament, Old Testament, Testament, Nouveau Testament, Avrcien, Vetus Testamentum, vetus novum, nuovo testament, gospels, evangelium secundum, mathhewm, gospel of St. Matthew, epistle to the romans, acts of the apostles, proverbs, psalms ecclesistes.

SEARCH MATCHING PROBLEM•1IF DEALING WITH COLLABORATIVE INPUTS TO DATA

•2Air Force Base•3air force base•4AFB•5A.F.B.•6A. F. B.

35

MULTIPLE MEANINGS: AMBIGUITY•1What is a “tank”?

•2weapon•3oil storage tank•4fish tank•5part of toilet unit

•6May need context to resolve•7Use of jargon and codes in groups

INDEXING TYPES •1Hierarchical•2Network (lateral)•3Subject headings•4Key words•5Syntactic languages•6Phrases•7Natural Language

HIERARCHICAL•1Single location in tree•2Precise object (definition of item)•3e.g. 1, 1.1, 1.2, 2.1, 2.1.1•4e.g. outlines, MDSE. files•5Minimum coding size, difficult to do and use •6Rigid, difficult to evolve / adapt

NETWORK (LATERAL LINKS)•1Single Location in network•2Precise relationship (definition of link)•3e.g. book index, citation index•4e.g. Hypertext (in theory, not practice)•5Lack of global view

CITATION INDEX I•1Index specific to a time like last year or last five years

•2Source of data all professional papers published during that time frame (on line version for all years recorded).

36

•3Papers published in refereed journals and conferences

CITATION INDEX IIFor a paper published at any time in the past and referenced

by papers published in that time frame, the results is a list of which papers referenced the earlier papers.

•1Uses reference list at back of papers to create an “inverted index”

CITATION INDEX PROPERTIES•1Most papers cease to be referenced five to ten years after publication

•2Most important contributions to science are papers reference ten and twenty years after publication

•3Some fields have significant number of papers never referenced (e.g. math is 30%)

CITATION INDEX UTILITY•1Useful for finding recent work in a given field defined by important papers in the field.

•2Useful for evaluating the contribution of individual scientists over the long term.

•3Takes 3 to 5 years after publication to begin to assess importance of individual paper.

SUBJECT HEADINGS•1Fixed categories, no structure, categorical scale•2Precise definitions of bin in which to fit items: Problem: book on “Use of integral equations in Physics”

Open choice as to file objects in multiple headings but not commonly done except through lateral linkages (e.g. cross references)

KEY WORD AND COORDINATE SYSTEMS•1Fixed key words•2Free key words•3Multiple keys for single object•4Coordinates for properties

37

•5e.g. tall, medium, or short

38

SYNTACTIC LANGUAGES: Tagged Descriptors•1Qualified keys, associated subcategories•2Mix of key word with attached hierarchies

•3e.g. tank.weapon or tank.petroleum•4e.g. steak.beef, steak.lamb, steak.veal

•5(type of meet).(source)

SYNTACTIC LANGUAGES: Faceted Index•1Separate dimensions to represent item

•2Like the many facets of a diamond•3softness, style, color, grade•4reference: author, title, source

•5Very common in science and commerce•6leather, wine, metal alloys, plastics

•7Both objective and subjective dimensions

MIXED INDEX EXAMPLE•1UNIVERSAL DECIMAL CLASSIFICATION

•2341.67:623.454.8(094.2)•3341.67: Disarmament, limitation and control of weapons of

mass destruction•4623.45: Ammunition, pyrotechnic devices, weapons of mass

destruction •5623.454.8: Active rays, atomic nuclear (thermo) weapons•6094.2: Historical sources (09), International treaties•7Why is nuclear weapons a subcategory of ammunition?

NATURAL LANGUAGE: Phrases•1Short phrases, titles

•2chapter/book titles, paper titles•3e.g. KWIC index

•4Often poor because done by original authorKWIC INDEX

•1Key Word In Context•2Alphabetical listing of all titles permuted word by word•3Words like of, the, a, an etc. not used.

•4Acceptance of Groupware Systems.•5Groupware Systems. Acceptance of•6Systems. Acceptance of Groupware

39

NATURAL LANGUAGE: Abstracts, Paragraphs•1Long enough to distinguish the single item uniquely from all others in the collection of items•2Good when searchers and writers of abstracts are experts in domain

•3English ambiguity a major problem

INDEXING EFFECTIVENESS I Relevant Non Relevant

Retrieved A BNot Retrieved C D

Precision = A / (A+B)Recall = A / (A+C)Specificity = D / (B+D)

Total Size Dependent (D)

Search Efficiency = (Recall)x(Specificity)

INDEXING EFFECTIVENESS IICannot have both high precision and high recall with

qualitative information and even very complex quantitative databases (e.g. all known plastics, metal alloys, etc.)

•1Indexing methods represent compromise between these two opposing objectives.

INDEXING EFFECTIVENESS III•1Too much recall leads to information overload and/or more effort for user

•2Too much precision leads to loss of relevant material and poor quality result for user

INDEXING EFFECTIVENESS IV•1Timeliness (updating and currency)•2Accuracy•3Completeness (All data in database)•4Form/structure of data useful (e.g. summary, raw data)

•5Depth and Breath (depth easier for novices, breath better for experts)

40

INDEXING EFFECTIVENESS V•1Discrimination & Resolution / Density•2Subjective / Objective•3Adoption & Acceptance•4Historical relevance•5Integration (analysis tools, other sources, etc.)

EVALUATION DIMENSIONS IINDEX TYPE AMBIGUITY EXPRESSIVE CONCISEHIERARCHICAL LOW LOW HIGHNETWORKSUBJECTSFIXED KEYSFREE KEYSTAGGED DESCRIPTORSFACETED INDEXESPHRASESNATURAL LANGUAGE HIGH HIGH LOW

EVALUATION DIMENSIONS IIINDEX TYPE

RETRIEVAL SELECTION ADOPTION EFFORT EFFORT EFFORTHIERARCHICAL LOW HIGH HIGHNETWORKSUBJECTSFIXED KEYSFREE KEYS LOWTAGGED DESCRIPTORSFACETED INDEXESPHRASESNATURAL LANGUAGE HIGH LOW HIGH

IDEAL USE OF INDEX TYPES I•1HIERARCHICAL

•2Macro concepts, well structured area (table of contents)

•3NETWORK•4Micro concepts, structured relationships (hypertext document)

•5SUBJECTS•6Macro concepts, structured objects (professional papers)

41

IDEAL USE OF INDEX TYPES II•1FIXED & FREE KEYS

•2Micro concepts, unstructured, evolving (book index, files)

•3Free if area is evolving•4Fixed when creators know more than users

IDEAL USE OF INDEX TYPES III•1TAGGED DESCRIPTORS & FACETED INDEXES

•2Micro, structured factors, well understood dimensions (e.g. materials, commodities)

•3PHRASES•4Macro, semi-structured (message titles, paper titles, book titles, etc.)

•5NATURAL LANGUAGE•6Macro, unstructured (abstracts)

ZIPF’S LAW•1Log frequency of terms in the English language (Y axis) are linear with log of rank order (X axis).

•2Product of frequency and rank order raised to fixed power is a statistical constant

•3Non log plot often called “draw down curve” and used in other fields.

SAMPLE Zipf List IWord Rank Frequency R x Fthe 1 68,315 68,315of 2 35,716 71,432and 3 27,856 83,568to 4 26,760 107,040a 5 22,744 113,720in 6 21,108 126,648that 7 11,188 78,316is 8 10,978 87,824was 9 10,499 94,491it 10 10,010 100,100

42

Sample Zipf List IIWord Rank Frequency R x Fsir 195 452 88,140it's 196 452 88,592why 197 451 88,847asked 198 448 88,704give 199 446 88,754once 200 443 88,600

Sample Zipf list IIIWord Rank Frequency R x Fusually 400 239 95,600tax 500 167 83,500ideas 800 128 102,400proved 1,170 88 102,960sections 2,146 49 105,154flames 5,070 17 86,190cultures 7,020 11 77,220

OBSERVATIONS ON LANGUAGE•1Order: Easiest to say to hardest to say (amazing collective evolution of language)

Can be derived from Classical Information Theory by minimizing total length of codes to represent message. Done years later after Zipf’s empirical studies.

CONSEQUENCES•1Led to Zipf’s Principle of Least Human Effort•2Human Behavior and the principle of least human effort: an introduction to human ecology, George Kngsley Zipf, 1949, 1972.

•3Concepts extended to many other areas of human and biological activity.

SEMANTIC BALANCE I•1Speaker’s economy in possessing a vocabulary of one

word with m different meanings.•2Auditor’s (listener) economy in possessing a vocabulary

of m different words with one distinctive meaning.•3Conflict in language and cause of “ambiguity.” A

collection of m words will have more than m meanings.

43

SEMANTIC BALANCE IIFirst 1,000 words (after initial 500) have like 6,000 meanings,

meaning they are used much more often than later words. (Ave. = 6)

•1First 20,000 words have like 40,000 meanings (Ave. = 2)

•2Suitable packet (unit of words) size can fit harmonic series

•3Compromise and tradeoffs in human affairs

Zipf RELATIONSHIPS•1Form of equation from: Generalized Harmonic Series•2r x f ^ (1/p) = C•3r = rank f = frequency•41 >= p >=0 •5for English p ~ 1, but specialized vocabularies differ

slightly.•6Distribution of star types provides insight into

evolution

ZIPF’s STUDIES I•1Number of retail establishments vs. population•2Diversity (number of different establishments) vc. population

•3Vocabulary of NY Times vs. ordinary paper (constant a lot larger)

•4Tool layout for craftsperson (furniture makers, shoe makers, watchmakers, etc.)

ZIPF’s STUDIES II•1Frequency of marriage vs. distance in blocks (before cars)•2Visitors to national parks vs. distance of home•3Size of cities (derived from agricultural model) before last

few decade•4Number and diversity of establishments vs. size of

population•5Biomass (frequency x weight) in pond (closed ecological

system) constant among species in food chain.

SYSTEMATIC KEY WORD LIST DEVELOPMENT I•1Analyze body of literature (significant sample) in application domain

•2Compare order to normal English and highlight

44

significant differences•3Concordance study useful for any writers

•4High frequency useless: too many matches

45

SYSTEMATIC KEY WORD LIST DEVELOPMENT II•1Low frequency useless: too few matches•2Use middle frequency terms different from normal English

Do correlation studies of two, three, four, and five terms together (within same paragraph, sentence) to seek clusters of matches.

PREPARING THE SAMPLE•1Remove useless words (“Exclusion list) from initial

sample•2Illustration of “a” words from such a list:

a about above after afterall afterward again against all almost already also although always am an and another any anyone anything are as at

•1Remove suffixes: (e.g. age als ant ary) from all words in sample

SEARCH APPROACHES I•1Scanning: Broad, top level, term lists•2Browsing: Up down, laterally, following interest, iterative, backup and start over

•3Target Searching: Specific target objective, need structure of data, categorization rules and understanding

SEARCH APPROACHES II•1Exploring: Objective is learning the nature of the database and contents.

•2Wandering: Creative stimulus, random patterns•3Both are a significant design challenge

SUBJECTIVE HUMAN INDEXING•1Spatial Indexing (e.g. piles of paper)•2Event Indexing (e.g. calendars)•3Color & Form Indexing (e.g. tabs, marks)•4Categorical (e.g. priorities)

46

SEARCH PROCEDURES I 1 Perceiving a need to find information. 2 Formulating the request for information.

3 Selection of the source for the information.

SEARCH PROCEDURES II 4 Specifying the search strategy. 4.1Browse the index, provide numbers 4.2Expand and contract levels 4.3Mark multiple choices (or) 4.4Provide for: or (novice), and, not

conditions, also distance conditions

SEARCH PROCEDURES III 5 Carrying out the search (mechanics). 5.1Show number of hits dynamically 5.2Browse the hit list 5.3Mark, Nest, Backup, Merge, Restart

SEARCH PROCEDURES IV 6 Evaluating the results 6.1Update index or list. Determine

relevance 6.2Saving, printing, viewing,

categorization 6.3Merging, reordering, clean up, store strategy

End of Lecture 1-5 Set

47