Presenter: Anuj Tewari Course: CS260 Instructor: John Canny Based on work by Indrani Medhi
Jan 14, 2016
Presenter: Anuj TewariCourse: CS260
Instructor: John CannyBased on work by Indrani Medhi
Outline: Text-Free User Interfaces
Introduction
Initial ethnography and iterative rapid prototyping
Initial evaluation
Optimal audio-visual representation
Methodology
Discussion
Outline: Text-Free User Interfaces
Introduction
Initial ethnography and iterative rapid prototyping
Initial evaluation
Optimal audio-visual representation
Methodology
Discussion
Illiteracy
1-2 billion illiterate population in the world.
98% live in developing countries.
India’s rate of literacy estimated at ~60%.
Many labeled “literate” are semi-literate with low functional literacy.
Photo: Indrani Medhi
Two residents of a slum in Jayanagar, Bangalore
Target Users
Women from Bangalore slums
Informal sector domestic workers
Income range: INR 800-2500 (US$20-60) per month
Illiterate or semi-literate
Most have never seen a PC (those who have seen have never touched)
Photo: Indrani Medhi
initial explorations before text-free UI work
NGO Partner
Stree Jagruti Samiti
Head: Geeta Menon Small NGO Works with several
Bangalore slums Focus on children’s
rights, women’s empowerment, fair wages for domestic laborers
Geeta Menon leading a SaturdayStree Jagruti Samiti meeting
Photo: Indrani Medhi
How can an application UI be converted into one that is usable by non-literate users?
The Problem
Outline: Text-Free User Interfaces
Introduction
Initial ethnography and iterative rapid prototyping
Initial evaluation
Optimal audio-visual representation
Methodology
Discussion
Initial EthnographyTVs, radios, CD players common; some mobile phones; no PCs
Established slums – some with 30-year history in city; concrete housing
Subjects most concerned about…
Jobs Healthcare Education
Nimmi interviewing a subject
Photo: Indrani Medhi
Text-Free UI?
Design principles:
Pen or touch interface
Liberal use of imagery○ No text○ Semi-abstracted
cartoons
Voice annotation
Aggressive use of mouse-over functionality
Consistent help iconNaukri.com for domestic labourers?
Nouns vs. VerbsGoal is to achieve exact association between image and word.
Without action cues drawings tend to be interpreted as nouns.
With action cues, drawings tend to be interpreted as verbs.
Phenomenon well-understood by cartoonists and diagram designers.
“Kitchen sink” or “washing dishes”?
Text-Free UI Details
“Pot” or “cooking”?
Cultural Differences
Goal is to achieve as global a representation as possible.
Abstracted representations are very dependent on culture.
Minor misrepresentations can result in major confusion.
An urban family of three?
Text-Free UI Details
Religious Differences
Goal is still to achieve as global a representation as possible.
Religious differences can be the strongest of cultural differences.
There is no “neutral” culture or religion.
Increasingly general representations for time indication
Text-Free UI Details
From when to when?
Quirks of Non-Literacy
Various degrees of literacy
Total illiteracy Numbers, but not alphabets Alphabets, but not words Words, but not sentences Semi-literate, but not fluent
Many who are non-literate can read numbers
Accustomed to currency Able to do basic arithmetic Not necessarily 100%
accurate
Illiteracy doesn’t necessarily mean inability to read numbers.
Text-Free UI Details
Text-Free UI?
Original design Revised design
Text-Free UI
Design principles:
– Pen or touch interface
– Liberal use of imagery• No text (but numbers OK)• Semi-abstracted cartoons
– Aggressive use of mouse-over functionality
– Voice feedback
– Consistent help icon
Screenshot of text-free job search
Other Domains
Principles transfer to other applications:
Text-free healthcare informationText-free maps
Outline: Text-Free User Interfaces
Introduction
Initial ethnography and iterative rapid prototyping
Initial evaluation
Optimal audio-visual representation
Methodology
Discussion
Questions
1. Can non-literate users use traditional text-based user interfaces at all?
2. Do the proposed design principles for text-free user interfaces allow non-literate users to use computers, and to what extent?
3. Which of the design principles make the most difference for a text-free UI?
Initial Evaluation
Experimental Set-UpThree versions:
Text-based Text-free without help
icon Text-free with help
icon
Subjects: Illiterate or semi-
literate○ (no literate subjects)
No PC experience
Subject grouping: Five-women group x 2 Individual x 4
Total time up to one hour per session
Task: For a friend who is unemployed, find the best-paying job in her neighborhood.
Measured: Task completion
(yes/no) Time required (second) Number of prompts
required (n)
All comments recorded; some trials video-recorded.
Initial Evaluation
Quantitative Results
Text-based UIs were completely unusable.
Obvious, but first formal such test in literature
With text-free UI, 30% were able to complete task.
(However, not enough participants for statistical significance.)
Tabulation of initial results
Initial Evaluation
Qualitative ResultsStrong preference for text-free principles, in particular…
Voice annotation Help icon
Collaborative use more successful
Subjects eager to engage, once comfortable with set up
Various barriers to technology: fear, lack of awareness, lack of cognitive model
Initial Evaluation
Indrani and a subject during initial evaluation
Questions
1. Can non-literate users use traditional text-based user interfaces at all?
2. Do the proposed design principles for text-free user interfaces allow non-literate users to use computers, and to what extent?
3. Which of the design principles make the most difference for a text-free UI?
Initial Evaluation
NO
Yes, but onlypartially; morework required
Hypothesis:- Voice feedback- Imagery - Help
Outline: Text-Free User Interfaces
Introduction
Initial ethnography and iterative rapid prototyping
Initial evaluation
Optimal audio-visual representation
Methodology
Discussion
Text-Free UI
Design principles:
– Pen or touch interface
– Liberal use of imagery• No text (but numbers OK)• Semi-abstracted cartoons
– Voice feedback
– Aggressive use of mouse-over functionality
– Consistent help icon
From initial round of research…
Screenshot of text-free job search
Goal of StudyDetermine what types of icon
representations are best for UIs for non-literate users.
Audio: ○ With voice annotation○ Without voice annotation
Visual:○ Photorealistic video○ Hand-drawn animation○ Photograph○ Hand-drawn image○ Text
Domain of Testing
Desired characteristics of domain:
Admits to visual representations, Consists of information which subjects are not exposed to on a
daily basis, Has universally recognized meaning, Offers a range of cognitive complexity, Offers a range of visual complexity, and Can be applied to a practical purpose.
Healthcare / medical imagery 13 medical symptoms: multimodal in nature, involving elements of
color, temperature, and some requiring motion spanning
DrawingPhotograph AnimationVideo
Ensure consistency across representations…
Creation of Representations
Tested Representations
DrawingPhotograph AnimationVideo Text
Each with and without voice annotation (total 10 representations),
for 13 medical symptoms
Experimental Set-UpRepresentations:
One of 10 types Evenly distributed
among subjects All 13 symptoms per
subject
Subjects: Illiterate or semi-literate
○ (no literate subjects) No PC experience
Size: Individual; n = 200 20 for each
representation
Total time up to one hour per session
Threefold task: Speak phrase that first
comes to mind when shown a representation.
Explain symptom representation is intended to convey
Enact or depict the intended symptom (as explained by tester), for analysis
Measured: Accuracy (correct,
close, wrong) Response time
(seconds)
All comments recorded
Voice annotation critical, with dominant effect
• 30% quicker response time with audio
Among cases with audio, hand-drawn representations (both animation and static imagery) most accurately recognized
• But, effect not dramatic
Among cases without audio, animation most accurately identified
• 23% more accurate than static drawing
ACCURATE RESPONSES (out of 20)
5.955.45
7.35
5.9
2.65
9.759.3 9.65 9.3 9.45
0
2
4
6
8
10
12
STATIC PHOTO ANIMATION VIDEO TEXT
AVERAGE RESPONSE Without AUDIO AVERAGE RESPONSE With AUDIO
Quantitative Results
Number of accurate responses (out of 20)for various representations
Qualitative Results
Test / text anxiety of subjects Subjects uncomfortable with test-like environment of studies
Short-term category conditioning Priming for “aches” over other types of symptoms when they
appeared in sequence
Richer information is not necessarily better understood overall Photorealism contains clutter; associated with actual instance
Unrelated Context-laying activity changes value of static versus dynamic imagery
Socio-economic conditions correlated with cognition Those with less formal education had difficulty processing
visuals and audio at the same time; confused by multi-modal interface
Outline: Text-Free User Interfaces
Introduction
Initial ethnography and iterative rapid prototyping
Initial evaluation
Optimal audio-visual representation
Methodology
Discussion
Initial Ethnography
Open-ended interviews Various combinations: focus
groups and individuals Topic: everything and anything Follow leads around hardships
Structured interviews Focus on themes from open-
ended interviews Prepare questions Be willing to deviate
Goal: Information, intuition, rapport
An open-ended interview in a subject’s home
Methodology
Time and attention critical!
Photo: Indrani Medhi
Iterative PrototypingBuild initial prototype, based on intuition from interviews.
Iterate prototype and test. Maintain informality Work with small groups
○ 2-4 people
Prompt whenever necessary Record every comment
○ Video-record, if possible (but, high procedural overhead)
Listen to every comment Modify prototype
Refine until steady state.Testing a prototype
Methodology
Photo: Indrani Medhi
Participative Design
Modify standard methods for non-literate partners.
Encourage storytelling.
Use aids that can be physically manipulated.
Be prepared for long, counseling-like sessions.
Methodology
Eliciting descriptions of medical symptomswith physical design aids
Photo: Indrani Medhi
Subject enacting medical symptoms
Formal EvaluationBase on techniques from courses, textbooks
Plan according to standard practices
Avoid direct counterfactuals Place task in a story Use friend or neighbor as
protagonist, but not subject Chavan’s “Bollywood method”
Be flexible on… Location (avoid labs) Formal instructions Etc.
Methodology
Video: Archana Prasad
Text-Free UI History
Initial principles
Optimal AVrepresenta-
tion
Full-contextvideo
Text-Free UIfor mobiles
Psychology of non-literate
users
Text-Freehealthcareinformation
Online site(babajob.com)
Practical Systems
UIResearch
OtherResearch
Completed Ongoing FutureKey:
UIauthoring
tool
Semi-automatedcartooning
Discussion questions
Is it possible to replicate the same study on a mobile? How would the process/guidelines for design be different?
What might be some other areas where text-free UI could have potential? How?
What are some of the best tricks to eliminate designer biases from the design process?