WLSACONVERGENCE SUMMIT
A PILOT STUDY OF AN INSPECTION FRAMEWORK FOR AUTOMATED USABILITY GUIDELINE REVIEWS OF MOBILE HEALTH APPLICATIONSJING XU, DEPARTMENT OF
COMPUTER SCIENCE UNIVERSITY OF MASSACHUSETTS LOWELL
A pilot study of an Inspection Framework for Automated Usability Guideline Reviews of Mobile Health
ApplicationsJing Xu, Xiang Ding, Ke Huang, Guanling Chen
Department of Computer ScienceUniversity of Massachusetts Lowell
Motivation: importance of usability
The importance of usability
from http://www.userlab.com/evaluations.html
4
Motivation: fast delivery mode
http://mhealthwatch.com
Motivation: negligible gap in the future
Smartphone penetration in US
from http://www.edisonresearch.com/2014-smartphone-ownership-demographics/
Motivation: stricter usability requirements for mHealth Apps
Human Aging and Web Use
from http://www.nngroup.com/articles/usability-for-senior-citizens/
Definition: According to Jakob Nielsen
…usability is:
Learnability
Efficiency
Memorability
Satisfaction
Error prevention
http://www.nngroup.com/articles/usability-101-introduction-to-usability/
1. Empirical (users are involved)
Three types of Usability testing:
2. Formal (Analytical Approaches)
Three types of Usability testing:
Usability Experts Forms, Models
Three types of Usability testing:
3. Inspection (Heuristic Evaluation)
Usability Experts Heuristics & Guidelines
from: https://blog.sensortower.com/
Top 25 ios App update frequency as of Apr 2014
Challenges I: fast iteration VS cost
Challenges II: diversity of hardware VS scalability
Android Device Fragmentationfrom: http://opensignal.com
Proposed Solution:
Automated guideline reviewsHeuristic set for the unique domain (mobile health apps)Translate heuristics into quantitative metricsAn automated app crawler
Talk Outline
• Specialized heuristics for mobile health apps
• The translation
• Methods
• Conclusion
Specialized Heuristics:
• Web applications
• Assistive robotics
• Collaborative systems
• Mobile games
• Game playability
• …Nielsen’sHeuristics
SpecializedHeuristics
Problems found
Specialized Heuristics for Mobile Health Apps (HIMSS):• Simplicity
• Naturalness
• Consistency
• Forgiveness and feedback
• Effective use of language
• Efficient interactions
• Efficient information presentation
• Preservation of context
• Minimum cognitive overload
Talk Outline
• Specialized heuristics for mobile health apps
• The translation
• Methods
• Conclusion and Future Works
Translation:
Heuristic 1Heuristic 2
.
.
.
.Heuristic 9
Heuristic 1Heuristic 2
.
.
.
.Heuristic 9
Task Level
Screen Level
Widget Level
Three Levels of MetricsHIMSS Heuristics for Mobile Health Apps
Task Level Metrics
• Number of steps
• Number of data inputs
• Pattern of user sequences
• Average operation distance
• Operations that require longer time
• Operations that consume more resources
Screen Level Metrics
• Information content
• Layout content
• Visual noises
• Number of Medical terms
• Number of ListViewItem count
• Number of MenuItem count
• Number of scrollable groups
Individual Widget Level Metrics
• Widgets leading to significant UI changes
• Default value and one-click delete
• Color, size and contrast ratio of text
• No inappropriate usage of red color
Talk Outline
• Specialized heuristics for mobile health apps
• The translation process
• Methods
• Conclusion
Methods:
CGGM of Mobile Apps
A state machine model of UI transitions
A state is a description of the structure of all equivalent screen views
S iS i
A transition is an UI interaction
CGGM of Mobile Apps
The description of the structure of a screen view
11 22
33 44
55 66
77
88
99
1010
1111
1212
1313
1414
CGGM of Mobile AppsCore of CGGM: use structure equivalence to
identify a unique state
Avoid state explosion Capture core functions Concise yet comprehensive
Now, we have the automated app crawler.
? What’s next
Raw metrics:
• State (UI) transitions• Response time of a transition• Location, size, type of widgets• Color, size and count of text• Words• Count of ListView Items• Count of Menu Items• Scrollable views
Raw Metrics to Metrics at Three Different Levels:
State Transitions Task Level
State Screen Level
Group Widget Level
The translation is based on the state machine.
Raw Metrics to Metrics at Three Different Levels:
Task Level
Definition of “task” based on the state
machine
Operation, pattern, data input
measurements based on tasks
Raw Metrics to Metrics at Three Different Levels:
Screen (State) Level
Space Usage
Space Complexity
Space Efficiency
Consistency
Inefficiency:
79 options9 options / screen
Diabetes Plus
Complexity:
ColorsColor contrastFontSpace usagetext countNumber of function groups
Diabetes Diary
Raw Metrics to Metrics at Three Different Levels:
Individual Widget Level
Properties: color, font, size
Adapt to different devices
Interaction efficiency
Efficiency:
Widget in group of size 1
Widgets associated with interactions that cause significant UI changes
Talk Outline
• Specialized heuristics for mobile health apps
• The translation process
• Methods
• Conclusion
To Conclude:
An automated inspection system cannot capture all usability problems
An assistive toolkit to help reduce the workload of human experts
Enforce a minimum bar for the usability of mobile health applications
Further adapted to different health domains and populations
Thanks!
WLSACONVERGENCE SUMMIT
www.wirelesshealth2014.org