Human Factors in Voting Systems John O’Hara IEEE Usability-Accessibility Working Group Chair HFES Voting System Task Force Chair Advisory Board Meeting.

Human Factors in Voting SystemsHuman Factors in Voting Systems

John O’HaraIEEE Usability-Accessibility Working Group Chair

HFES Voting System Task Force Chair

Advisory Board MeetingWashington, DC

October 16, 2002

TopicsTopics

• Background

• Technical perspective

• The voting system

• Human factors engineering of voting systems

• Recent HFE standards activities for voting systems

• Beyond the current standards

BackgroundBackground

• Voting technology is changing

• FEC revised its voting system standards to accommodate new technology

• FEC’s standards do not completely address human factors aspects of voting systems

• IEEE standard will strengthen the treatment of usability and accessibility

• FEC is sponsoring R&D to more completely address voting system human factors and usability

Voting Technology TrendsVoting Technology Trends

• DRE and optical scan only technologies increasing in usage

from IEEE Spectrum, 2000

Technical PerspectiveTechnical Perspective• Complex systems

– socio-technological systems that integrate hardware, software, and human resources to achieve a specific goal

– high-risk, high reliability– often serve critical infrastructure missions, such as defense/security,

transportation, health care, and energy

Technical PerspectiveTechnical Perspective• Performance is addressed through systems engineering and

defense-in-depth design– high design standards– redundancy and diversity for system elements– analysis of likely failures and design for recovery from them– personnel handle normal and failure conditions through procedures and

training

How is Human Performance Addressed?How is Human Performance Addressed?

• Human performance is addressed through human factors engineering (HFE)

• Knowledge about human characteristics and capabilities is developed in many scientific disciplines– physiology, psychology, and sociology/anthropology

• Human factors is:– Research specifically devoted to developing knowledge about the

integration of human and machine systems– Application of that knowledge to the design of systems and equipment

General Human Factors ActivitiesGeneral Human Factors Activities• Goals

– highly reliable system and human performance– simplicity and standardization– error tolerance (minimization, recognition, and recovery)

• Requirements analysis (tasks, users, etc.)

• Detailed design of equipment– design characteristics and functions– application of HFE guidelines

• Development of supporting infrastructure– instructions, procedures, and training

• Test and evaluation

• Performance monitoring and feedback

Is Systems Engineering an Appropriate Model for Voting Systems?

Is Systems Engineering an Appropriate Model for Voting Systems?

• Voting systems involve the integration and coordination of hardware, software, and human resources to achieve a critical mission

• The system elements all contribute to mission success although they are “loosely coupled” and distributed– e.g., good ballot design software does not guarantee that good ballots

will be designed by election officials using the software

• Human performance is vital to success of the mission

Yes

The Voting SystemThe Voting System

Election Management System

Control Subsystem

Vote Recording System

Reporting System

from Draft IEEE P-1581

• System elements are the infrastructure that accomplish the voting process

The Voting SystemThe Voting System• Election Management System

– administrative database – ballot generation – election programming– ballot validation

• Control Subsystem– equipment preparation– pre-delivery testing – tests at the polling place– opening the polling place– enabling a ballot– error recovery– closing the polling place– polling place reports

• Vote Recording Subsystem– precinct voting – on-site absentee voting, early

voting, and state-wide voting– remote absentee voting

• Reporting Subsystem– polling place reports– precinct reports– consolidated reports– system audit log– access to election data

User GroupsUser Groups

Vendors

Voters

Test Labs

Voting Officials

PrecinctWorkers

MaintenancePersonnel

Lesson Learned from theAnalysis of Complex System Failure

Lesson Learned from theAnalysis of Complex System Failure

• Overall performance is a function of the integrated system– failure is often the result of the interaction of system elements– difficult to focus on one component of the system

• Quick fix is often to throw technology at a problem– the technology “halo” effect– technology itself does not enhance or detract from human performance – often old problems are solved, but new problems are created

Broader and UnanticipatedEffects of Technology

Broader and UnanticipatedEffects of Technology

• Changes in the roles and responsibilities of people working in the system

• Changes in the ways people accomplish their tasks

• Changes in the organizational structure– e.g., need for an Information Technology Department to manage

software configuration, etc.

• Digital technology evolves much more quickly than other technologies and organizations need to be able to respond

Voting System FailureVoting System Failure

• Since voting system elements are “loosely coupled,” standards aimed at one aspect of the system may not guarantee high-reliability of the whole system

• A comprehensive, hierarchical approach to voting system human factors is needed– individual standards addressing system elements (and probably

different user groups)– an overall standard or guidance document that ties the individual

elements together

Supporting Human PerformanceThrough Human Factors

Supporting Human PerformanceThrough Human Factors

• Thorough analysis of human tasks– for all significant user groups in the system– address primary and secondary tasks

• Human-centered equipment design– enable user to devote full attention to the task at hand – to minimize secondary tasks (minimize attention shifts)– make the equipment as transparent as possible

• Some key principles– design reflects basic human cognitive and physiological capabilities– positive guidance for appropriate task behavior– high-degree of simplicity, standardization, and consistency– error tolerance (minimize error through design and allow errors to be

recognized and recovered from)

Supporting Human Performancein the System

Supporting Human Performancein the System

• Comprehensive test and evaluations– “usability tests” to discover and address unanticipated issues– verifications against human factors design standards– validation that desired system performance criteria are met

• Well-designed procedures and instructions– all user groups– for all user tasks

• Training, familiarization, and help

• Defense-in-depth system design– error tolerance across all system elements– multiple barriers establish layers of defense against errors and failures

VSS Usability AppendixVSS Usability Appendix

• Provided a systems engineering view of the voting system usability

• Guidance on key aspects of voting system human factors engineering was provided– equipment design– infrastructure support, such as familiarization and training

• Fairly high-level and narrow focus

IEEE P-1583Usability-Accessibility Sections

IEEE P-1583Usability-Accessibility Sections

• Builds on the VSS Usability Appendix

• Focus– equipment design requirements– voter interface– standards verification for compliance

Topics AddressedTopics Addressed

• 6.5 Usability and Accessibility– 6.5.1 General Principles – 6.5.2 Overall Design and Layout of the Voting Location– 6.5.3 Voter Identification and Authentication– 6.5.4 Ballot and Information Presentation– 6.5.5 Voter Input and Response– 6.5.6 Navigation and Interaction with Ballots– 6.5.7 Preventing and Minimizing Voter Errors– 6.5.8 Help and System Failure– 6.5.9 Voter Familiarization and Training

• 7.5 Usability and Accessibility Testing

Further IEEE Standard DevelopmentFurther IEEE Standard Development

• Increase level of detail for voting equipment human-system interfaces– touch screens– cursor control devices and behavior

• Improve accessibility standards

• Develop standards verification criteria

Beyond the Current StandardsBeyond the Current Standards

• Expanded the scope of the standards to the whole voting system– treatment of other technological approaches to voting equipment– test and evaluation methods beyond verification– ballot preparation and testing– voter familiarization and training– equipment setup

• General guidance on “how to” – we have standards, e.g., for equipment design, but do not offer

guidance to designers on how (methods) to accomplish good HFE

• Tie it all together– standards and guidance should be presented as an integrated whole

Beyond the Current StandardsBeyond the Current Standards

VendorsTest Labs

PrecinctWorkers

Voting Officials

TestingHandbook

TestingHandbook

BallotDesign

Handbook

BallotDesign

Handbook

EquipmentDesign

Handbook

EquipmentDesign

Handbook

BallotDesign

Standard

BallotDesign

StandardEquipmentStandard

EquipmentStandard

VotingSystem

HFE

VotingSystem

HFE

TestingStandard

TestingStandard

VoterTraining

Handbook

VoterTraining

Handbook

VoterTrainingStandard

VoterTrainingStandard

Vendors