WH2014 Session: A pilot study of an inspection framework for automated usability guideline

WLSACONVERGENCE SUMMIT

A PILOT STUDY OF AN INSPECTION FRAMEWORK FOR AUTOMATED USABILITY GUIDELINE REVIEWS OF MOBILE HEALTH APPLICATIONSJING XU, DEPARTMENT OF

COMPUTER SCIENCE UNIVERSITY OF MASSACHUSETTS LOWELL

A pilot study of an Inspection Framework for Automated Usability Guideline Reviews of Mobile Health

ApplicationsJing Xu, Xiang Ding, Ke Huang, Guanling Chen

Department of Computer ScienceUniversity of Massachusetts Lowell

Motivation: importance of usability

The importance of usability

from http://www.userlab.com/evaluations.html

4

Motivation: fast delivery mode

http://mhealthwatch.com

Motivation: negligible gap in the future

Smartphone penetration in US

from http://www.edisonresearch.com/2014-smartphone-ownership-demographics/

Motivation: stricter usability requirements for mHealth Apps

Human Aging and Web Use

from http://www.nngroup.com/articles/usability-for-senior-citizens/

Definition: According to Jakob Nielsen

…usability is:

Learnability

Efficiency

Memorability

Satisfaction

Error prevention

http://www.nngroup.com/articles/usability-101-introduction-to-usability/

1. Empirical (users are involved)

Three types of Usability testing:

2. Formal (Analytical Approaches)

Three types of Usability testing:

Usability Experts Forms, Models

Three types of Usability testing:

3. Inspection (Heuristic Evaluation)

Usability Experts Heuristics & Guidelines

from: https://blog.sensortower.com/

Top 25 ios App update frequency as of Apr 2014

Challenges I: fast iteration VS cost

Challenges II: diversity of hardware VS scalability

Android Device Fragmentationfrom: http://opensignal.com

Proposed Solution:

Automated guideline reviewsHeuristic set for the unique domain (mobile health apps)Translate heuristics into quantitative metricsAn automated app crawler

Talk Outline

• Specialized heuristics for mobile health apps

• The translation

• Methods

• Conclusion

Specialized Heuristics:

• Web applications

• Assistive robotics

• Collaborative systems

• Mobile games

• Game playability

• …Nielsen’sHeuristics

SpecializedHeuristics

Problems found

Specialized Heuristics for Mobile Health Apps (HIMSS):• Simplicity

• Naturalness

• Consistency

• Forgiveness and feedback

• Effective use of language

• Efficient interactions

• Efficient information presentation

• Preservation of context

• Minimum cognitive overload

Talk Outline

• Specialized heuristics for mobile health apps

• The translation

• Methods

• Conclusion and Future Works

Translation:

Heuristic 1Heuristic 2

.

.

.

.Heuristic 9

Heuristic 1Heuristic 2

.

.

.

.Heuristic 9

Task Level

Screen Level

Widget Level

Three Levels of MetricsHIMSS Heuristics for Mobile Health Apps

Task Level Metrics

• Number of steps

• Number of data inputs

• Pattern of user sequences

• Average operation distance

• Operations that require longer time

• Operations that consume more resources

Screen Level Metrics

• Information content

• Layout content

• Visual noises

• Number of Medical terms

• Number of ListViewItem count

• Number of MenuItem count

• Number of scrollable groups

Individual Widget Level Metrics

• Widgets leading to significant UI changes

• Default value and one-click delete

• Color, size and contrast ratio of text

• No inappropriate usage of red color

Talk Outline

• Specialized heuristics for mobile health apps

• The translation process

• Methods

• Conclusion

Methods:

CGGM of Mobile Apps

A state machine model of UI transitions

A state is a description of the structure of all equivalent screen views

S iS i

A transition is an UI interaction

CGGM of Mobile Apps

The description of the structure of a screen view

11 22

33 44

55 66

77

88

99

1010

1111

1212

1313

1414

CGGM of Mobile AppsCore of CGGM: use structure equivalence to

identify a unique state

Avoid state explosion Capture core functions Concise yet comprehensive

Now, we have the automated app crawler.

? What’s next

Raw metrics:

• State (UI) transitions• Response time of a transition• Location, size, type of widgets• Color, size and count of text• Words• Count of ListView Items• Count of Menu Items• Scrollable views

Raw Metrics to Metrics at Three Different Levels:

State Transitions Task Level

State Screen Level

Group Widget Level

The translation is based on the state machine.

Raw Metrics to Metrics at Three Different Levels:

Task Level

Definition of “task” based on the state

machine

Operation, pattern, data input

measurements based on tasks

Raw Metrics to Metrics at Three Different Levels:

Screen (State) Level

Space Usage

Space Complexity

Space Efficiency

Consistency

Inefficiency:

79 options9 options / screen

Diabetes Plus

Complexity:

ColorsColor contrastFontSpace usagetext countNumber of function groups

Diabetes Diary

Raw Metrics to Metrics at Three Different Levels:

Individual Widget Level

Properties: color, font, size

Adapt to different devices

Interaction efficiency

Efficiency:

Widget in group of size 1

Widgets associated with interactions that cause significant UI changes

Talk Outline

• Specialized heuristics for mobile health apps

• The translation process

• Methods

• Conclusion

To Conclude:

An automated inspection system cannot capture all usability problems

An assistive toolkit to help reduce the workload of human experts

Enforce a minimum bar for the usability of mobile health applications

Further adapted to different health domains and populations

Thanks!

WLSACONVERGENCE SUMMIT

www.wirelesshealth2014.org