Usability modeling and measurement
Post on 20-May-2015
3127 Views
Preview:
DESCRIPTION
Transcript
Usability Modeling and Measurement
Philip Lew
www.xbosoft.com
Agenda
• Introduction– The importance of usability – Specifics for the web / mobile
• What is Usability?• Usability Modeling and Measurements• Case study• Conclusion• Next Steps
Importance of Usability
• Usability is important especially for web applications (SaaS, and websites) – And now especially mobile applications
• Most prevalent development delivery model today
• Without good usability:– Users will leave the applications– For mobile, if they can’t learn in 30 seconds,
they won’t come back
Why is Usability Important?
Web and Mobile have Changed the Terrain
• Business models have changed– Instead of paying
upfront and ‘owning’ the software
– Pay as you go, pay by subscription
• Behavior and expectations have changed
Many Alternatives• Depending on the
type of application– Users could leave
and go elsewhere– Simply not use your
mobile app (if only providing alternative access to a main application)
Thin Client DeploymentMost Prevalent Development Delivery Model Today
• Mobile and Cloud are singing the same song• Service delivery versus product delivery
What is Usability
In relation to qualityIn relation to user experience
General Thoughts on Usability
• Understandability• Learnability• Operability• Attractiveness• Navigation• Responsiveness-performance• Efficiency
Usability in Quality Standards• How is usability
defined?– Standard
definitions– Many others
who say similar things
• Usability– Component
of quality as listed in many standards
Standard DefinitionISO 9126-1(2000)
The capability of the software product to be understood, learned, used, and attractive to the user, when used under specified conditions.
ISO 9241-11(1998)
The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.
IEEE 600.12(1990)
The ease with which a user can learn to operate, prepare inputs for, and interpret outputs of a system or component.
Seffah [33]
Usability Model Comparisons
Current Research / Usability Standards
Kappel et al., The Discipline of Systematic Development of Web Applications, 2003, John Wiley
and Sons.
Current Research – Usability Standards
Quality
Usability
Satisfaction
ISO9126-1
ISO 9241-11
IEEE 610
IEEE XXXYISO 25010
Bigger Picture Quality From ISO point of view
ISO 25010CMMI
Usa
bilit
y
Usa
bilit
y
Usability as a Key Characteristic of Product Quality
Source: ISO 25010
Source: ISO 25010
What is Usability-Effect of the Software Product
Degree to which specified users can achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.
Source: ISO 25010
What is Usability-EffectIn Actual Usage
• Effectiveness– The degree to which specified users can achieve specified goals with
accuracy and completeness in a specified context of use.• Efficiency
– The degree to which specified users expend appropriate amounts of resources in relation to the effectiveness achieved in a specified context of use.
– NOTE Relevant resources can include time to complete the task, materials, or the financial cost of usage.
• Satisfaction– The degree to which users are satisfied in a specified context of use.
Satisfaction is further subdivided into sub-characteristics:• Likability (cognitive satisfaction)• Pleasure (emotional satisfaction)• Comfort (physical satisfaction)• Trust
Usability in Actual Usage
• User role• Objective• Task• Environment• Domain• …
specified users
specified goals
specified context of use
What else can you think of?
Defining Usability For Your Organization
• Attributes expressed hierarchically• Any number of sub-levels is OK• Achieving the sub-attributes=achieving the high level
attribute -measurable
Quality
Attribute 1
Characteristic 1 Characteristic 2 Characteristic n
Subcharacteristic 1 Subcharacteristic 2 Subcharacteristic n
Attribute 2 Attribute 3 Attribute 1
Usability
Let’s Define UsabilityFrom the Product Viewpoint
Usability
Save Location
Characteristic 1 Characteristic 2 Characteristic n
Subcharacteristic 1 Subcharacteristic 2 Subcharacteristic n
Attribute 1 Attribute 2 Attribute n
UnderstandableNavigation
Control Stability Previous-Next
Home Location Ease of finding
Bigger Picture on Usability
• Can be measured from the design point of view or of the product
• Can be measured ‘in-use’ with real users
Usa
bilit
y
Usa
bilit
y
Defining Usability from an Effect-Real usage Point of View
Quality
Errors
Effectiveness Efficiency Characteristic n
Accuracy Completeness Subcharacteristic n
Attribute 2 Attribute 3 Attribute 1
Usability
Satisfaction
Other Potential Attributes Measurements for ‘in use’ Usability
• Effectiveness– Completion rates– Error rate– Help usage
• Efficiency– Task time– Backtracking
• Learnability– Learning rate– Task time deviation
Developing a CombinedUsability Model
Application Design and Environment
User Behavior and Activity
Measurable and quantifiable factors
Measured ‘in-use’ Usability
External Quality Requirements (for Shopping Cart Entity) 1 Usability 1.1 Understandability 1.1.1 Icon/label ease to be recognized 1.1.2 Information grouping cohesiveness 1.2 Learnability 1.2.1 ……………………………………………………….. 1.3 Operability 1.3.1 Control permanence 1.3.2 Expected behaviour of Controls 2 Content Quality 2.1 Content Suitability 2.1.1 Basic Information Coverage 2.1.1.1 Line item information completeness 2.1.1.2 Product description appropriateness 2.1.2 Coverage of other Contextual Information 2.1.2.1 …………………………………………………………
Example of Product Quality Model with Attributes
(Operability in ISO 25010)
Measurable Attributes
• Attribute name• Description and purpose• How to measure• What is measured• Measurement/Calculation• Range (min, max)• Objective• Current
Once you have a model (what you are going to measure), then you start doing IT!
Usability Measurement
27
Attribute Scale How Measure orCalculation
Objective Current
Help completeness
Percent ofMenu items with help
Comparemenus andhelp items
% 90% 40%
Ease ofaccess
Keystrokes tofind/use a feature/function/information
Sample 50items
% 3 12
Consistency Number locations for same button
Examinemenus anddoc.
integer 1 5
Accuracy Number reportederrors
Collect from log files
Integer or % <5 10
Usability Measurement Methods
Let’s get started
Logging
Heuristic Evaluation
Labs
Focus groups
WalkThroughs
Satisfaction Surveys
Current Research-Summary of Usability Evaluation Methods
Usability LoggingMeasurement and Data Collection
• Identify users by using session ID to identify a unique user.
• Iteratively insert code into the application
• Collect data• Analyze the data for each
attribute in different dimensions and aggregations
• Determine the need for further calculations and what attributes to measure further
• Revise the data we are collecting, adding or decreasing granularity
Satisfaction Surveys
The process:1. Calculate the usability score (satisfaction) of version X2. Do the survey3. Change to version X.1 – make changes to the
software directly correlated to the usability factors to either increase or decrease the usability score
4. Do the survey again 5. See if differences made change impact the survey
results
Notes on Satisfaction and Usability
• Satisfaction is a subjective feeling dependent on many things other than usability:– A user can be highly
satisfied but the application with low usability.
– An application can be highly usable (high usability) but the user is not satisfied!
Highly usablesoftware
My passworddoesn’t work
don’t havewhat I want I’m unsatisfied
I’m satisfied!
!low usability
software
Finished my work today
Nice weather today
External Quality Requirements Measure EI value P/GI value Global Quality Indicator 61.97%
1 Usability 60.88% 1.1 Understandability 83% 1.1.1 Icon/label ease to be recognized 100% 1.1.2 Information grouping cohesiveness 66% 1.2 Learnability 51.97% 1.2.1 ……………………………………………… … 1.3 Operability 49.50% 1.3.1 Control permanence 100% 1.3.2 Expected behaviour 50% 2 Content Quality 63.05% 2.1 Content Suitability 63.05% 2.1.1 Basic Information Coverage 50% 2.1.1.1 Line item information completeness 2 50% 2.1.1.2 Product description appropriateness 50% 2.1.2 Coverage of other Contextual Information 76.89% 2.1.2.1 ……………………………………………….. … 2.1.2.2 Return policy information completeness 33%
Sample Evaluation
Healthcare SoftwareHeuristic Evaluation
Conclusion
• Usability is an abstract concept• Defining usability is different for each
organization• Need a model for your organization• The model is the foundation of what to
measure• Once you can measure, then you can evaluate
and improve
Next Steps
• Produce an action plan– What usability attributes are important to your
organization?• Develop a model
– What data can you collect/Which technique can you use
• Maybe some elements of the model drop out-can’t be measured that easily
– Start collecting and developing benchmarks• Discuss with your manager/team
Thanks
Questions and Answers
top related