Design and Design and Evaluation Evaluation Human Factors Psychology Human Factors Psychology Dr. Steve Dr. Steve
Dec 16, 2015
Design and Design and EvaluationEvaluation
Human Factors PsychologyHuman Factors Psychology
Dr. SteveDr. Steve
Human Factors Design Human Factors Design ActivitiesActivities
Design product/system interfacesDesign product/system interfaces Modify designs/correct design Modify designs/correct design
flawsflaws Safety activities (warnings, Safety activities (warnings,
analyses)analyses) Develop training Develop training
programs/materialsprograms/materials
Cost/Benefit AnalysisCost/Benefit Analysis
Costs• HF personnel salaries • Extended time to product release• Analysis tools (software, video)• Prototyping • Increased development cost
Benefits• More effective product• Safer product• Cheaper to change design early in development• Higher sales• Fewer liability suits
What other costs and benefits of HF activity in product/system design can you think of?
System Development and System Development and HF ActivitiesHF Activities
• Stage 1. Front-End Analysis*• user analysis• function analysis• preliminary task analysis• environmental analysis• system specifications
• Stage 2. Conceptual Design*• function allocation• support conceptual design
• Stage 3. Iterative Design and Testing*• task analysis• interface design & prototyping• heuristic evaluation• cost/benefit analysis of alternatives• workload analysis (simulation & modeling)• safety analysis• usability testing
• Stage 4. Design of Support Materials• develop manuals
• Stage 5. System Production• Stage 6. Implementation and Evaluation
• experiments to evaluate system• Stage 7. System Operation/Maintenance
• monitor system performance• Stage 8. System Disposal
* Topics covered in class
User-Centered DesignUser-Centered Design
1. Early focus on the user and tasks2. Empirical measurements
• questionnaires, usability studies, data collection3. Iterative design
• prototyping of interface4. Participatory design
• user input to design, preferences, SLUGs
Sources for Design DataSources for Design Data
• Data Compendiums•Engineering Data Compendium: Human Perception and Performance
• Human Factors Design Standards• MIL-STD-1472D (military anthropometry, ergonomic specs)• ANSI/HFES-100 (VDT specs)• ANSI/HFES-200 (software specs)• OSHA standards
• General HF Principles/Guidelines• Textbooks (by topic: interface design, aviation, software)• Research Journals (Human Factors, Ergonomics, Applied Ergonomics)• Handbooks (Handbook of Human Factors)• Conferences (CHI, HFES, IEA)• Workshops (U of Mich)
Front-End AnalysisFront-End AnalysisAnswers the following questions:
1. Who are the users?• Consumers, trainers, maintainers, etc.
2. What functions are performed by system?• Allocate functions to machine or person
3. What are the environmental conditions where system will be used?• Noise, temperature, pressures, etc.
4. What are users’ preferences/requirements?• Color, functions, shape, etc.
5. What are constraints to design of the system?• Space, regulations, costs, resources
6. What are the HF criteria for design solutions?• Shorter training time, less accidents, higher performance
Front-End AnalysisFront-End AnalysisUser AnalysisUser Analysis
Who are the intended users?• Age• Experience level• Intellectual level• Anthropometrics (size & shape)• Strength• Culture & Language• Disabilities (sight, hearing, mobility)
Why does the drive- thru ATM have Braille on its controls? What are the characteristics of DVD player consumers?
Front-End AnalysisFront-End AnalysisFunction/Preliminary Task Function/Preliminary Task
AnalysisAnalysis
Function Analysis: What are the basic functions of the system?Example: The function of a wheel barrow is to move objects to the work site.
Preliminary Task Analysis: What are the jobs, duties, tasks, and actions that user will be performing?Example: Tasks to be performed with a wheel barrow include loading it with dirt/building materials, transporting, mixing, and dumping materials, cleaning, maintaining, and storing wheelbarrow.
Front-End AnalysisFront-End Analysis Preliminary Task AnalysisPreliminary Task Analysis
Preliminary Task Analysis Methods:• Interviews• Focus Groups• Observation• Questionnaires
Used prior to design to learn how intended users perform their jobs.
What method(s) would you use if you were designing a new library cataloguing system?
Front-End AnalysisFront-End AnalysisEnvironment AnalysisEnvironment Analysis
Where will the tasks be completed and under what conditions?• Indoors/outdoors• Temperature/humidity• Lighting levels• Noise levels• Vibration• Gravity• Special clothing/gear requirements
• How would the design of a radio transmitter differ if the intended users were researchers in Antarctica?• Would a standard vending machine work on the space station? If not, how would you design a vending machine for that purpose?
Front-End AnalysisFront-End AnalysisUser Preferences & Req’sUser Preferences & Req’s
When different design options are available, must determine the intended users preferences and needs (perhaps determined through focus groups).
User preference issues include:• What components are automated• Color features/aesthetics• Balance between performance and preference• What makes other products popular/unpopular• What can be made adjustable and what cannot• Feasibility of user’s demands
Conceptual Design Conceptual Design ActivitiesActivities
Function AllocationFunction Allocation
Things Humans Do Well Ability to:• detect small amounts of visual or acoustic energy• perceive patterns of light or sound• improvise and use flexible procedures• store very large amounts of information for long periods and to recall relevant facts at the appropriate time• reason inductively• exercise judgment
Things Machines Do WellAbility to:• respond quickly to control signals, and to apply great force smoothly and precisely• perform repetitive, routine tasks• store information briefly and then to erase it completely• reason deductively, including computational ability• handle highly complex operations, I.e., to do many different things at once
Fitt’s List (1951)
Function Allocation: the assigning of tasks to humans or machine taking into account human vs. machine capabilities and limitations
Is Fitt’s list still appropriate 50+ years later?
Conceptual Design ActivitiesConceptual Design ActivitiesDecision/Action & Functional Flow Decision/Action & Functional Flow
AnalysesAnalysesDecision/Action and Functional Flow Analyses: Procedures for identifying and depicting the sequence of functions, actions, and decisions made by the system (user included as part of system).
Example: Programming your VCR to record a show Thursday night at 7:30 on Fox Network
Press “Program”
“use up & downarrows to
indicate selection”
Accessinputmenu
Determine When show
airs
Hit arrowto indicate
day
VCR acceptsInput, moves to next field
Press“Enter” button
Hit arrowto indicate
time
VCR acceptsInput, moves to next field
Press “Enter” button
Hit arrowto indicate
channel
VCR acceptsInput
Press“Enter” button
“Program completeSelect another
program or pressEnd to finish”
Yes or No
Press“End”
Yes
No
(See homework assignment)
Conceptual Design Conceptual Design ActivitiesActivities
Typical Design MeetingTypical Design Meeting
BrainstormingStory-Boarding
Iterative Design and Iterative Design and TestingTesting
Task Analysis (detailed)Task Analysis (detailed)
Purpose of Detailed Task Analysis is to identify:• User goals and activities• Tasks required to achieve goals• Conditions under which task is performed• Expected outcomes of tasks• KSA’s needed to perform tasks• Equipment needed to perform tasks• Complex decision making, problem solving, diagnoses*• Complex rule structures dependent on situation*
* Part of Cognitive Task Analysis
Task Analysis MethodsTask Analysis MethodsInterviewsInterviews
Interviews: Questioning subject matter experts to get them to explain what they do on the job.
Tries to answer questions such as:• How do you perform the task?• Why do you perform the task?• Under what conditions do you perform the task?• What happens before and after performing the task?• What are the consequences of performing or not performing the task?
Advantage: simple to useDisadvantage: SMEs may not be good at describing tasks
Task Analysis MethodsTask Analysis MethodsObservationObservation
Observation: Watch and take notes of people performing the task
Suggestions: • Be unobtrusive• Use video if possible and with approval• Review observations with SMEs
Advantages: See what workers do, not what they say they do.Disadvantages: Does not capture cognitive processes.
Task Analysis MethodsTask Analysis MethodsThink-Aloud Verbal ProtocolThink-Aloud Verbal Protocol
Think, think, think!
Think-Aloud Verbal Protocol: Worker state what they are thinking as they perform the task
• Concurrent – verbalize as performing the task• Retrospective – verbalize what they were doing as they view a videotape of themselves• Prospective – verbalize as they imagine performing a hypothetical task
Advantage: Can get info regarding thought processes and decisions of experts performing the task.
Disadvantage: Doing concurrently can interfere with the task, and doing retrospectively relies too heavily on memory
Task Analysis MethodsTask Analysis MethodsHierarchical Task AnalysisHierarchical Task Analysis
Hierarchical Task Analysis: Graphical (chart or diagram) depiction of task analysis
Advantage: Allows to conceptualize task sequencesDisadvantage: May not accurately represent true parallel nature of task
Example: Table 3.5 from Wickens, Gordon, & LiuHierarchical task analysis for using a lawnmowerStep 1. Examine Lawn a. Make sure grass is dry
b. Look for any objects laying in the grassStep 2. Inspect Lawnmower
a. Check components for tightness1. Make sure grass bag handle is securely fastened to grass bag support2. Make sure grass bag connector is securely fastened to bag adapter3. Make sure the deck cover is in place4. Check for any loose parts (such as oil cap) and blade
Iterative Design and Iterative Design and TestingTesting
Link AnalysisLink Analysis
Link Analysis: Shows the relationships between components of a system (e.g., communication, control, movements)
For determining:• layout of related workstations• placement of controls• communication patterns• frequency of equipment use
Advantage: Graphic depiction of relationships among system featuresDisadvantage: Shows frequency of use, not importance, and may vary by situation
Link Analysis of CIC of USS Louisville
Iterative Design and Iterative Design and TestingTesting
Flow DiagramsFlow Diagrams
Operational Sequence Diagram: Graphically depicts the sequence and timing of activity, and shows the interactions between individuals and equipment
Advantage: Connects actions to time and componentDisadvantage: Does not lend itself easily to interface design
Diagram for a collision avoidance system
Iterative Design and Iterative Design and TestingTesting
Interface Design PrinciplesInterface Design Principles
1. Provide a good conceptual model2. Make things visible3. Use natural mappings4. Provide feedback
Iterative Design and Iterative Design and TestingTesting
Interface Design PrinciplesInterface Design Principles
Provide a good conceptual model: when things are designed to be consistent with our mental models of how they should work (expectancies) then it is easier to predict what will happen when we use them.
In this metronome, the tempos are arranged around a circle with a knob and arrow in the middle. Most people would expect to twist the knob until the arrow points to the desired tempo. Actually the arrow is a button that you push if you want the tempo to increase and the knob is not a knob at all.
Iterative Design and Iterative Design and TestingTesting
Interface Design PrinciplesInterface Design Principles
Make things visible: It is easier to operate a system when the features are clearly visible
Starting this gas pump is more difficult than it sounds because the start button is hidden among the other sea of information and does not “pop out” at you as a start button either figuratively or literally. Additionally, the use of red usually signals something other than start.
Iterative Design and Iterative Design and TestingTesting
Interface Design PrinciplesInterface Design Principles
Use natural mappings:Actions should correspond to
the system’s state.To make something go up,
the action should be up, to make it go right, the action should be to the right
This stovetop makes good use of natural mappings. The burners are arranged in the same pattern as the controls.
Iterative Design and Iterative Design and TestingTesting
Interface Design PrinciplesInterface Design Principles
Provide feedback: Feedback is important because it allows one to see whether
the intended actions have taken place. (like hourglass icon on computer software)
(read “Set Phasers to Stun” in Casey book for another example).
In this example the feedback is confusing. The light on the bottom signifies that the smaller quantity of coffee was requested. The light does not go on when the larger quantity is requested.
Iterative Design and Iterative Design and TestingTesting
Interface Design PrinciplesInterface Design Principles
Other Design Principles (Norman, 1992):• Simplify the structure of tasks• Make alternative actions and knowledge of results visible• Make it easy to determine what actions are possible• Make it easy to evaluate current system state• Exploit the power of constraints (remove possibility of error)• Design for easy error recovery (undo)• When all else fails, standardize.
Iterative Design and Iterative Design and TestingTesting
PrototypingPrototyping
Prototypes or Mockups:Used to test features of the interface prior to production - may be anything from crude cardboard cut-outs to realistic models
Example of Rapid Prototyping Software used to model different phone interfaces
Iterative Design and Iterative Design and TestingTesting
Evaluative StudiesEvaluative Studies
Types of evaluative studies:• Cost/Benefit Analysis for Design Alternatives• Trade-Off Analyses• Safety Analysis• Workload Analysis• Simulation & Modeling
Mathematical Modeling
Evaluative Studies Evaluative Studies Simulation & ModelingSimulation & Modeling
CognitiveModeling
Biomechanical Modeling
Modeling programs:•MicroSAINT•Mannequin•COMBIMAN•HUMANCAD
Evaluative Studies Evaluative Studies Usability TestingUsability Testing
Usability Testing: Ease of use (user-friendly)
May test for:• Learnability• Efficiency• Memorability• Errors• Satisfaction
Final Test and EvaluationFinal Test and EvaluationField StudiesField Studies
Design: between vs. within subjectsTest Participants: user populationProximal Measures:
• satisfaction• usability• performance• errors
Distal Measures:• manufacturing costs• personnel costs• number of accidents/disability claims
Alternatives to HF Design and Alternatives to HF Design and DevelopmentDevelopment
TrainingTo use system capabilitiesMethods to make task safer
Modifying Current EquipmentEquipment ReplacementErgonomic Aids
Wrist padsFoot stoolsAnti-fatigue mats
Job Rotation (and other admin approaches)Vary tasksRest periods