Observation and Usability Studies Ericka Brunson Dorothy Hirsch Mollee March Joanna McCloud Denise Tiller
Feb 23, 2016
Observation and Usability Studies
Ericka BrunsonDorothy HirschMollee MarchJoanna McCloudDenise Tiller
Focus on human behavior & actions
Part of mixed-methods approach
Naturalistic: how people really behave
What they have in common
Test validity of other findings
Study specific processes or behaviors
Access behaviors of those unable to communicate
Observation studies can
Three types of studies
Participant Observation
Reactive Observation
Unobtrusive Observation
Study reference transactions
Analyze patron/librarian behavior
Study patron use of resources and tools
Library uses
a thorough study over time
conducted in a natural environment
when the observer becomes part of the group being studied
rarely used by libraries because of expense in time and money
the basis of most cultural anthropological/ethnology group studies
Participant observation is
Only studies behaviors, not cognition
Prone to research bias/misconceptions
Can be expensive, both in time and money
Ethical considerations
Limitations
How information is obtained
What is done with the information
Do no harm: emotionally mentally socially economically
Ethical issues
Are you watching me right now…or am I watching you?
Reactive observation
A “real world” observational research method that studies individuals and groups of people in a natural, controlled setting
Participants are aware that they are being studied.
What is reactive observation?
Behavior reflects aspects of social desirability
Observations can be manipulated to fit the research design
Reactive observation positives
Individuals are reacting to observation
External validity
Reactive observation negatives
Continuous monitoring
Time allocation
Common methods of reactive observation
Involves studying the behavior of participants and recording as much as possible.
Works best in organizational settings such as:the workplaceeducational instituteswhen monitoring nutrition (exercise, food
intake, etc.)
Continuous monitoring
The “Hawthorne Effect” states that “workers react to the attention they are getting from the researchers and in turn their productivity increases.”
The “Hawthorne Effect”
Done at random
Researcher will choose the time and place to record activities before the researcher’s presence is known
Time allocation
Descriptive
Inferential
Evaluative
Variables to consider in evaluating observations
Subjects are unaware that they are being observed and researchers do not intrude on the study.
Unobtrusive measures
Behavior trace studies
Disguised field observations
Types of measures
Natural behavior of subjects
Reduces bias
Unobtrusive positives
Researcher has little control over type of data collected
Not always a usable method
Subjects unaware of data being gathered
Other ethical issues
Unobtrusive negatives
Tests system performance with typical users
Many library applications
Quick, easy, inexpensive
Usability studies - a bright idea
Exploratory – early stage using screen shots and paper visuals
Assessment – mid-point focusing on the actual product
Verification – final stage to ensure product meets standards
Comparison – used in any stage to compare design options
Four types of usability studies
Usefulness
Ease of learning
Ease of use
User satisfaction
Usability studies test
Card sort
Prototypes
Usability tasks
Focus groups
Individual interviews
Methods used to collect data
Type of study Exploratory
Purpose Test JoCo Library Children’s Web site
Methods used Prototypes, usability tasks, focus group
Participants Eleven children ages 6 to 11
Examples of library usability studies
Johnson County Public LibrariesJoCoKids Usability Study
April 25, 2009
Type of study Assessment
Purpose Test new web site
Methods used Prototypes, usability tasks, focus group
Participants Five mothers with children under age 6
Johnson County Public Libraries6 by 6 Usability StudySeptember 1-2, 2009
Step 1Let participants explore the prototype web site and record comments
Step 2Usability tasks
6 by 6 usability study
Type of study Verification
Purpose Verify usability of new paper application
Methods used Usability task
Participants 10 patrons
Johnson County Public LibrariesNew Paper Application Card Usability
StudyJune 16, 2009
Anschutz Library Renovation Project
2009-2010
Dec Emailed survey to faculty, staff and students
Mar-Sept Redesigned the main library floor and created the Learning Studio
Oct “Snapshot Day”
Beck, S.E. & Manuel,K. (2008). Observation and usability. In Practical research methods for librarians and information professionals. New York, N.Y.: Neal-Schuman Publishers, Inc.
Brown, L. & Trochim, W.M. (2006). Research methods knowledge base. Retrieved from Cornell University website: http://researchmethods.net.
Brown, L. (n.d.). Observational ield Research [Online tutorial]. Retrieved from Cornell University Library website: http://www.socialresearchmethods.net/tutorial/Brown/lauratp.htm.
Hendry, J. (1999). Other people's worlds: An introduction to cultural and social anthroplogy. New York, New York: Washington Square University Press.
Kumar, K. (1996). Using direct observation techniques [ CDIE Tips Series,4]. Retrieved from USAID Evaluation Publications website: http://pdf.usaid.gov/pdf_docs/PNABY208.pdf.
Lee, R.M. (2000). Introduction to unobtrusive methods. In Unobtrusive methods in social research, pp. 1-16.
Norlin, E. (2000). Reference evaluation: a three strep approach – surveys, unobtrusive observations, and focus groups. College and Research Libraries, 61(6),pp. 546-553.
Paul, B.D. (1953). Interview technigues and field relationships. In Kroeber, A.L. (1953). Anthropology Today: an encyclopedic inventory. Chicago, Illinois: University of Chicago Press.
Radin, P. (1966). The method and theory of ethnology: An essay in criticism[2nd ed.] New york and London, England: Basic Books. Reece, R.D. & Siegal, H. A. (1986). Studying people: A primer in the ethics of social research. Macon, Georgia: Mercer University Press.
Romanczyk, R.G. ,Kent, R.N., Diament, C., & O'Leary, D. (1973). Measuring the reliability of observational data: a reactive process. Journal of Applied Behavior Analysis, 6(1), 175-184.
Trochim, W.M. The research methods knowledge base, [2nd ed.]. Retrieved from: http://www.socialresearchmethods.net/kb/. Last revised: 10/20/2010.
Usability studies at Johnson County Libraries (n.d.) Retrieved from: http://jocolibrary.org/templates/JCL_InfoPage.aspx?id=3063&epslanguages=EN.
References