Computing for Social Needs Jennifer Mankoff UC Berkeley.

Computing for Social Needs

Jennifer Mankoff

UC Berkeley

Inspiration in the World: Finding the Right Combination

Hard, real problemsHard HCI problems

Low intuition about users Success hard to test Technology not always a good solution

Hard computer science problems

ExamplesWord prediction (+ & -)Augmented canes (+ & -)

OutlineApproaches to research in computing

for social needs (CSN)Example: DesignExample: MethodExample: ToolConclusions

Approaches To Research in CSNDesign: For usersMethod: For designers/evaluatorsTool: For programmers/designers

Approaches To Research in CSNDesign: For users

Identify need Investigate solutions Prototype, test & iterate

Method: For designers/evaluatorsTool: For programmers/designers

Approaches To Research in CSNDesign: For usersMethod: For designers/evaluators

Identify model or theory Test against circumstances or population Iterate

Tool: For programmers/designers

Approaches To Research in CSNDesign: For usersMethod: For designers/evaluatorsTool: For programmers/designers

Identify repeating need or use of technology

Abstract out Test for reusability

OutlineApproaches to research in computing

for social needs (CSN)Example: DesignExample: MethodExample: ToolConclusions

Design Example: NutritionNeed: Healthier dietsAssumptions Idea: Keep track of purchases, display

advice

Design Example: NutritionNeed: Healthier diets

Manage disease America’s weight problem Manage child health

Assumptions Idea: Keep track of purchases, display

advice

Design Example: NutritionNeed: Healthier dietsAssumptions

People don’t really know what they consume

Receipts contain enough information for us to estimate nutrition

Idea: Keep track of purchases, display advice

Design Example: NutritionNeed: Healthier dietsAssumptions Idea: Keep track of purchases, display

advice

Nutrition: Hard HCI ProblemsFormative evaluation: testing perception Interface designSummative evaluation in real-use

setting

Nutrition: Formative EvalSurvey shoppersBackground research

Nutrition: Formative EvalSurvey shoppers

Perceived calcium consumption Perceived need for supplements Calcium consumption in receipts

Background research

Nutrition: Formative EvalSurvey shoppersBackground research

Use of shopping receipts in bookkeeping Interest in nutrition % of time eating out Impact of coupons, advice on shopping

behavior

Nutrition: Hard HCI ProblemsFormative evaluation: testing perception Interface design

While at home Continual Peripheral

While shopping While entering data

Summative evaluation in real-use setting

“Was that ‘Apple cider’Or ‘Apple scraper’

Nutrition: Hard HCI ProblemsFormative evaluation: testing perception Interface designSummative evaluation in real-use

setting Measures change in awareness Measures change in behavior

Nutrition: Hard Computer Science ProblemsRecognition

OCR Who eats what Quantities, ingredients

Ambiguity

Nutrition: Hard Computer Science ProblemsRecognition

OCR Who eats what Quantities, ingredients

Ambiguity

Nutrition: Hard Computer Science ProblemsRecognitionAmbiguity

Resolving imperfect recognition automatically

Resolving imperfect recognition with user’s help

OutlineApproaches to research in computing

for social needs (CSN)Example: DesignExample: MethodExample: ToolConclusions

Method Example: Comparative AccessibilityNeed: Increased accessibility in all

interfacesAssumptions Idea: Develop metrics for interpreting

simulated testing results

Method Example: Comparative AccessibilityNeed: Increased accessibility in all

interfaces More inclusive Increase quality of life

Assumptions Idea: Develop metrics for interpreting

simulated testing results

Method Example: Comparative AccessibilityNeed: Increased accessibility in all

interfacesAssumptions

Can’t test every interface with every type of disability

Can simulate disability sufficiently for testing

Idea: Develop metrics for interpreting simulated testing results

Method Example: Comparative AccessibilityNeed: Increased accessibility in all

interfacesAssumptions Idea: Develop metrics for interpreting

simulated testing results

Comparative Accessibility: Hard HCI ProblemsCan a novice simulating disability give

feedback on an interface designed for experts in that disability?

How should heuristics include accessibility?

How do disabilities impact GOMS models?

How do disabilities impact Fitts’ law?

OutlineApproaches to research in computing

for social needs (CSN)Example: DesignExample: MethodExample: ToolConclusions

Tool Example: Mouse predictionsNeed: Access to any application Assumptions Idea: Recognize problems, predict

targets, and use that to make the mouse do the right thing

Tool Example: Mouse predictionsNeed: Access to any application

Equal access Increased independence

Assumptions Idea: Recognize problems, predict

targets, and use that to make the mouse do the right thing

Tool Example: Mouse predictionsNeed: Access to any applicationAssumptions

Low vision or motor impairment No access to application code Access to OS (e.g. app can be installed)

Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing

Tool Example: Mouse predictionsNeed: Access to any application Assumptions Idea: Recognize problems, predict

targets, and use that to make the mouse do the right thing.

Mouse predictions: Hard HCI problemsExisting motion models only account for

averagesExisting user models inaccurateUI for compensation unclear

Mouse predictions: Hard HCI problemsExisting motion models only account for

averages Minimum jerk model:

X(t) = X0 + (X0 – Xf) (154 - 65 - 103)

Fitts’ law: MT = a + b log(A/W)

Existing user models inaccurateUI for compensation unclear

Mouse predictions: Hard HCI problemsExisting motion models only account for

averagesExisting user models inaccurate

KLM extra cognitive cycles No model of fatigue

UI for compensation unclear

Mouse Prediction: Other Models Velocity

Thrashing ( = target)

Spasming

Overshooting

Other characteristics?

Mouse predictions: Hard HCI problemsExisting motion models only account for

averagesExisting user models inaccurateUI for compensation unclear

“Beat Fitts’ law” Feedback affects recognition

Mouse Predictions – UIs for CompensationGravity wells and area mouseMediation

Mouse Predictions: Hard Computer Science ProblemsRecognition

Account for feedback Account for fatigue

Ambiguity Better interfaces for multiple targets? Interface for multiple directions? Appropriate balance of control and

automation

OutlineApproaches to research in computing

for social needs (CSN)Example: DesignExample: MethodExample: ToolConclusions

ConclusionsPlenty of hard real problemsPlenty of hard HCI problemsPlenty of hard computer science

problemsResearch needed in designs, methods

& tools

Thank You

For More Information:

[email protected]

http://www.cs.berkeley.edu/~jmankoff

Tool Example: Reconstruction of Mismatched InterfacesNeed: Adaptation to any set of input

devices