ITIS 6010/8010 Usable Privacy & Security

ITIS 6010/8010ITIS 6010/8010Usable Privacy & Usable Privacy & SecuritySecurity

Dr. Heather Richter LipfordDr. Heather Richter Lipford

[email protected]@uncc.edu

AgendaAgenda

Evaluation (from last time)Evaluation (from last time) Ethics & IRBEthics & IRB Assignments updateAssignments update Chapter 2 & 3 discussionChapter 2 & 3 discussion

When to do When to do evaluation?evaluation? SummativeSummative

– assess an existing systemassess an existing system– judge if it meets some criteriajudge if it meets some criteria

FormativeFormative– assess a system being designedassess a system being designed– gather input to inform designgather input to inform design

Which you do depends on maturity of Which you do depends on maturity of prototypes and goals of evaluationprototypes and goals of evaluation

Same techniques work for bothSame techniques work for both

Evaluation techniquesEvaluation techniques

Feedback from expertsFeedback from experts– Discount usability techniques: heuristic evaluation, Discount usability techniques: heuristic evaluation,

cognitive walkthroughcognitive walkthrough Observe usersObserve users

– Think-aloud & Cooperative evaluationThink-aloud & Cooperative evaluation Talk to usersTalk to users

– Interviews & Focus groupsInterviews & Focus groups Survey usersSurvey users

– QuestionnairesQuestionnaires Test hypothesesTest hypotheses

– ExperimentsExperiments

Typical User StudyTypical User Study

Bring participants into a controlled Bring participants into a controlled setting (lab)setting (lab)

Introductions and consentIntroductions and consent Gather demographic data and give Gather demographic data and give

instructionsinstructions Ask participant to do a set of tasksAsk participant to do a set of tasks

– Prototype can be simulated or partially Prototype can be simulated or partially functionalfunctional

Observe and record behaviorObserve and record behavior Ask participant for feedback about Ask participant for feedback about

interfaceinterface

Many variationsMany variations

Show or demonstrate mockup, Show or demonstrate mockup, storyboard, screenshots and storyboard, screenshots and gather feedbackgather feedback

Observe or gather data about Observe or gather data about behavior in a natural settingbehavior in a natural setting

Can be multiple sessions or just Can be multiple sessions or just oneone

Evaluation planningEvaluation planning

Decide on techniques, tasks, materialsDecide on techniques, tasks, materials– What are usability criteria?What are usability criteria?– How much required authenticity?How much required authenticity?

How many people, how longHow many people, how long How to record data, how to analyze How to record data, how to analyze

datadata Prepare materials – interfaces, Prepare materials – interfaces,

storyboards, questionnaires, etc.storyboards, questionnaires, etc. Pilot the entire evaluation Pilot the entire evaluation

– Test all materials, tasks, questionnaires, etc.Test all materials, tasks, questionnaires, etc.– Find and fix the problems with wording, Find and fix the problems with wording,

assumptionsassumptions– Get good feel for length of studyGet good feel for length of study

General General RecommendationsRecommendations

Clearly identify evaluation goalsClearly identify evaluation goals Include both objective & subjective dataInclude both objective & subjective data

– e.g. “completion time” and “preference”e.g. “completion time” and “preference” Use multiple measures, within a typeUse multiple measures, within a type

– e.g. “reaction time” and “accuracy”e.g. “reaction time” and “accuracy” Use quantitative measures where Use quantitative measures where

possiblepossible– e.g. preference e.g. preference scorescore (on a scale of 1-7) (on a scale of 1-7)

Note: Only gather the data required; do so Note: Only gather the data required; do so with minimum interruption, hassle, time, with minimum interruption, hassle, time, etc.etc.

Performing the StudyPerforming the Study

Be well prepared so participant’s time is not Be well prepared so participant’s time is not wastedwasted

Describe the purpose of the evaluationDescribe the purpose of the evaluation– ““I’m testing the product; I’m not testing you”I’m testing the product; I’m not testing you”

Explain procedures without compromising Explain procedures without compromising resultsresults

Session should not be too long , subject can Session should not be too long , subject can quit anytimequit anytime

Never express displeasure or angerNever express displeasure or anger Data to be stored anonymously, securely, Data to be stored anonymously, securely,

and/or destroyedand/or destroyed

ConsentConsent

Why important?Why important?– People can be sensitive about this process and People can be sensitive about this process and

issues issues – Errors will likely be made, participant may feel Errors will likely be made, participant may feel

inadequateinadequate– May be mentally or physically strenuousMay be mentally or physically strenuous

What are the potential risks (there are What are the potential risks (there are alwaysalways risks)? risks)?

““Vulnerable” populations need special care & Vulnerable” populations need special care & consideration consideration – Children; disabled; pregnant; students (why?)Children; disabled; pregnant; students (why?)

More later on IRB…More later on IRB…

Now what do you do?Now what do you do?

Start just looking at the dataStart just looking at the data– Were there outliers, people who fell Were there outliers, people who fell

asleep, anyone who tried to mess up the asleep, anyone who tried to mess up the study, etc.?study, etc.?

Sort & prioritize the dataSort & prioritize the data Identify & summarize issues:Identify & summarize issues:

– Overall, how did people do?Overall, how did people do?– ““5 W’s” (Where, what, why, when, and for 5 W’s” (Where, what, why, when, and for

whom were the problems?)whom were the problems?) Compile aggregate results and Compile aggregate results and

descriptive statisticsdescriptive statistics

Making ConclusionsMaking Conclusions

Where did you meet your criteria? Where did you meet your criteria? Where didn’t you?Where didn’t you?

What were the problems? How serious What were the problems? How serious are these problems?are these problems?

What design changes should be made?What design changes should be made?– Update task analysis, scenarios, etc.Update task analysis, scenarios, etc.

Prioritize and plan changes to the Prioritize and plan changes to the designdesign

Modify prototypes and go againModify prototypes and go again

ExperimentsExperiments

A A controlledcontrolled way to determine impact of way to determine impact of design parameters on user experiencedesign parameters on user experience

Want results to eliminate possiblity of Want results to eliminate possiblity of chancechance

HypothesisHypothesis: What you predict will : What you predict will happenhappen– More specifically, the way you predict the More specifically, the way you predict the

dependent variable (i.e., accuracy) will dependent variable (i.e., accuracy) will depend on the independent variable(s)depend on the independent variable(s)

Types of VariablesTypes of Variables

Independent Independent – What you’re studying, what you What you’re studying, what you

intentionally vary (e.g., interface feature, intentionally vary (e.g., interface feature, interaction device, selection technique)interaction device, selection technique)

DependentDependent– Performance measures you record or Performance measures you record or

examine (e.g., time, number of errors)examine (e.g., time, number of errors) ControlledControlled

– Factors you want to prevent from Factors you want to prevent from influencing resultsinfluencing results

““Controlling” Controlling” VariablesVariables

Prevent a variable from affecting the Prevent a variable from affecting the results in any systematic wayresults in any systematic way

Methods of controlling for a variable:Methods of controlling for a variable:– Don’t allow it to vary Don’t allow it to vary

e.g., all malese.g., all males

– Allow it to vary randomly Allow it to vary randomly e.g., randomly assign participants to different e.g., randomly assign participants to different

groupsgroups

– Counterbalance - systematically vary it Counterbalance - systematically vary it e.g., equal number of males, females in each groupe.g., equal number of males, females in each group

The appropriate option depends on The appropriate option depends on circumstancescircumstances

ExampleExample

Do people complete operations faster Do people complete operations faster with a black-and-white display or a color with a black-and-white display or a color one?one?– Independent - display type (color or b/w)Independent - display type (color or b/w)– Dependent - time to complete task Dependent - time to complete task

(minutes)(minutes)– Controlled variables - same number of Controlled variables - same number of

males and females in each groupmales and females in each group– Hypothesis: Time to complete the task will Hypothesis: Time to complete the task will

be shorter for users with color displaybe shorter for users with color display

– HHoo: Time: Timecolorcolor = Time = Timeb/wb/w

Experimental DesignsExperimental Designs

Within Subjects DesignWithin Subjects Design– Every participant provides a score Every participant provides a score

for all levels or conditionsfor all levels or conditions Color B/WP1 12 secs. 17 secs.P2 19 secs. 15 secs.P3 13 secs. 21 secs....

Experimental DesignsExperimental Designs

Between SubjectsBetween Subjects– Each participant provides results for Each participant provides results for

only one conditiononly one condition Color B/WP1 12 secs. P2 17 secs.P3 19 secs. P5 15 secs.P4 13 secs. P6 21 secs....

ComparisonComparison

Within subjectsWithin subjects– More efficient: fewer trials and participantsMore efficient: fewer trials and participants– But need to avoid “order effects”But need to avoid “order effects”

e.g. seeing color then b/w may be different from e.g. seeing color then b/w may be different from seeing b/w then colorseeing b/w then color

Between subjectsBetween subjects– Simpler design & analysis because fewer Simpler design & analysis because fewer

order effectsorder effects– Often shorter, so easier to recruit Often shorter, so easier to recruit

participantparticipant– More subjects for same statistical powerMore subjects for same statistical power

Hypothesis TestingHypothesis Testing

Tests to determine differencesTests to determine differences– t-test to compare two meanst-test to compare two means– ANOVA (Analysis of Variance) to ANOVA (Analysis of Variance) to

compare several meanscompare several means– Need to determine “statistical Need to determine “statistical

significance”significance”

““Significance level” (p):Significance level” (p):– The probability that your null hypothesis The probability that your null hypothesis

was wrong, was wrong, simply by chancesimply by chance– p (“alpha” level) is often set at 0.05, or p (“alpha” level) is often set at 0.05, or

5% of the time you’ll get the result you 5% of the time you’ll get the result you saw, just by chancesaw, just by chance

Discount Evaluation Discount Evaluation TechniquesTechniques Basis:Basis:

– Observing users can be time-consuming Observing users can be time-consuming and expensiveand expensive

– Try to predict usability rather than Try to predict usability rather than observing it directlyobserving it directly

– Conserve resources (quick & low cost)Conserve resources (quick & low cost) Expert reviewers usedExpert reviewers used

– HCI experts interact with system and try to HCI experts interact with system and try to find potential problems and give find potential problems and give prescriptive feedbackprescriptive feedback

Example: Heuristic Example: Heuristic evaluationevaluation

3-5 experts in HCI view or interact with a 3-5 experts in HCI view or interact with a prototype.prototype.– May vary from mock-ups and storyboards to a working May vary from mock-ups and storyboards to a working

systemsystem They use high-level heuristics as guidelines, and They use high-level heuristics as guidelines, and

identify any problems they see. For example:identify any problems they see. For example:– Does the interface use natural and simple dialog?Does the interface use natural and simple dialog?– Does the interface provide good error messages?Does the interface provide good error messages?

Designers compile and summarize all the Designers compile and summarize all the problems and iterate.problems and iterate.

Where to get heuristics?Where to get heuristics?– http://www.useit.com/papers/heuristic/http://www.useit.com/papers/heuristic/– http://www.asktog.com/basics/firstPrinciples.htmlhttp://www.asktog.com/basics/firstPrinciples.html

Cognitive WalkthroughCognitive Walkthrough

Assess Assess learnabilitylearnability and usability through and usability through simulation of way simulation of way novicenovice users explore and users explore and become familiar with interactive systembecome familiar with interactive system

Experts walk through all steps in Experts walk through all steps in representative tasks, identifying trouble spots representative tasks, identifying trouble spots based on 4 questionsbased on 4 questions• Will users be trying to produce whatever effect action has?Will users be trying to produce whatever effect action has?• Will users be able to notice that the correct action is Will users be able to notice that the correct action is

available? (is it visible)available? (is it visible)• Once found, will they know it’s the right one for desired Once found, will they know it’s the right one for desired

effect? (is it correct)effect? (is it correct)• Will users understand feedback after action?Will users understand feedback after action?

Advantages & Advantages & DisadvantagesDisadvantages Fast and cheapFast and cheap Does not need working systemDoes not need working system Detailed, careful examination that Detailed, careful examination that

can cover entire interfacecan cover entire interface Problems are subjective – are Problems are subjective – are

they really usability problems?they really usability problems? Outcomes depend upon expertise Outcomes depend upon expertise

and experience of the reviewersand experience of the reviewers

For more info:For more info:

http://www.sis.uncc.edu/~richter/clhttp://www.sis.uncc.edu/~richter/classes/2006/6010/index.htmlasses/2006/6010/index.html

or or

http://www.sis.uncc.edu/~clatulip/Ihttp://www.sis.uncc.edu/~clatulip/ITIS6400/ITIS6400_Home.htmlTIS6400/ITIS6400_Home.html

Or take the course in the spring.Or take the course in the spring.

Ethics of working with Ethics of working with peoplepeople

Usability testing can be arduous; privacy Usability testing can be arduous; privacy is importantis important

Each person should know and Each person should know and understand what they are participating understand what they are participating in:in:– what to expect, time commitmentswhat to expect, time commitments– what the potential risks arewhat the potential risks are– how their information will be usedhow their information will be used

Must be able to stop without danger or Must be able to stop without danger or penaltypenalty

All participants to be treated with respectAll participants to be treated with respect

Attribution TheoryAttribution Theory

Studies why people believe that Studies why people believe that they succeeded or failed--they succeeded or failed--themselves or outside factors themselves or outside factors (gender, age differences)(gender, age differences)

Make sure participants do not Make sure participants do not feel that they did something feel that they did something wrong, that the errors are their wrong, that the errors are their problemproblem

Respecting your Respecting your participantsparticipants

Be well prepared so participant’s time is not wastedBe well prepared so participant’s time is not wasted Make sure they know you are testing software, not Make sure they know you are testing software, not

themthem Explain procedures without compromising resultsExplain procedures without compromising results Make them aware they can quit anytimeMake them aware they can quit anytime Make sure participant is comfortableMake sure participant is comfortable Session should not be too longSession should not be too long Maintain relaxed atmosphereMaintain relaxed atmosphere Never indicate displeasure or angerNever indicate displeasure or anger State how session will help you improve system State how session will help you improve system

(“debriefing”)(“debriefing”) Don’t compromise privacy (never identify people, Don’t compromise privacy (never identify people,

only show videos with explicit permission)only show videos with explicit permission)

IRBIRB

Institutional Review Board (IRB)Institutional Review Board (IRB) Federal law governs proceduresFederal law governs procedures Reviews all Reviews all researchresearch involving human involving human

(or animal) participants(or animal) participants Safeguarding the participants, and Safeguarding the participants, and

thereby the researcher and universitythereby the researcher and university Not a science review (i.e., not to asess Not a science review (i.e., not to asess

your research ideas); only safety & your research ideas); only safety & ethicsethics

http://www.research.uncc.edu/Comp/human.cfmhttp://www.research.uncc.edu/Comp/human.cfm

Ethics CertificationEthics Certification

Ethics is not just common senseEthics is not just common sense Training being standardized to ensure Training being standardized to ensure

even and equal understanding of even and equal understanding of issuesissues

Go get your certification: due Sept. Go get your certification: due Sept. 18!18!

http://www.research.uncc.edu/tutorial/index3.cfmhttp://www.research.uncc.edu/tutorial/index3.cfm

IRB @ UNCCIRB @ UNCC

http://www.research.uncc.edu/comp/human.cfmhttp://www.research.uncc.edu/comp/human.cfm

On-line tutorialOn-line tutorial GuidelinesGuidelines Consent procedures and template formsConsent procedures and template forms Protocol application formsProtocol application forms

IRB Protocol 101 TrainingIRB Protocol 101 Training– http://www.research.uncc.edu/comp/human_trng.cfmhttp://www.research.uncc.edu/comp/human_trng.cfm– 9/10, 9/11, 9/12, 9/18, 9/20 from 6-7pm9/10, 9/11, 9/12, 9/18, 9/20 from 6-7pm

AssignmentsAssignments

ScenarioScenario

Your target users work in a hospital. Confidentiality of patient Your target users work in a hospital. Confidentiality of patient data cannot be compromised. Different employees have data cannot be compromised. Different employees have different levels of clearance within the one system that controls different levels of clearance within the one system that controls all of the patient records. There are a limited number of public all of the patient records. There are a limited number of public workstations that are highly trafficked throughout the day. workstations that are highly trafficked throughout the day. Current practice at the hospital is that one worker logs in and Current practice at the hospital is that one worker logs in and often many people with different levels of clearance work under often many people with different levels of clearance work under that same account, even though they are not authorized to do that same account, even though they are not authorized to do so. Often, the workstation remains logged in between users, so. Often, the workstation remains logged in between users, thus an unauthorized user could gain access to patient records. thus an unauthorized user could gain access to patient records. In addition, passwords change on a monthly basis so it is more In addition, passwords change on a monthly basis so it is more convenient for the workers to just use the account that has convenient for the workers to just use the account that has already been logged in than try to recall their always changing already been logged in than try to recall their always changing password. Management insists that the passwords much password. Management insists that the passwords much change frequently to reduce the risk of a hacker viewing the change frequently to reduce the risk of a hacker viewing the confidential data.confidential data.

How to address security needs with passwords or other forms How to address security needs with passwords or other forms of authentication in this context?of authentication in this context?

My current scenarioMy current scenario

Your target users are students and faculty doing studies Your target users are students and faculty doing studies in the usability lab. This lab is a room with two Novell in the usability lab. This lab is a room with two Novell computers with special usability recording software on computers with special usability recording software on them. Access to the lab is controlled by 49er card. All them. Access to the lab is controlled by 49er card. All study personnel need to be able to access the study study personnel need to be able to access the study materials, but no one else should have access to those materials, but no one else should have access to those materials. Study materials include consent forms, materials. Study materials include consent forms, questionnaires, and instructions to give study questionnaires, and instructions to give study participants, both in digital and physical forms. participants, both in digital and physical forms. Additionally, on the computer are the application and Additionally, on the computer are the application and application data to be tested, as well as the digital application data to be tested, as well as the digital recordings of the study. An external hard drive contains recordings of the study. An external hard drive contains back up copies of all the recordings and application data. back up copies of all the recordings and application data. Most of the people in the lab will have Novell accounts, Most of the people in the lab will have Novell accounts, but not everyone.but not everyone.

How can we provide shared access to the study How can we provide shared access to the study materials? How can we prevent unauthorized people materials? How can we prevent unauthorized people from getting access to the study materials and records?from getting access to the study materials and records?

Usable Privacy & Usable Privacy & Security: An Security: An IntroductionIntroduction ““weakest link property” – attackers only weakest link property” – attackers only

have to exploit one error or vulnerabilityhave to exploit one error or vulnerability Sociotechnical system – complex system Sociotechnical system – complex system

of technologies and people/organizationsof technologies and people/organizations

So are people really the weakest link in So are people really the weakest link in security or privacy systems? How much security or privacy systems? How much is a self-fulfilling prophecy?is a self-fulfilling prophecy?

Are security and usability competing Are security and usability competing goals?goals?

The productThe product

The technologies and processes put in The technologies and processes put in place for security and privacy protectionplace for security and privacy protection

Why don’t they work?Why don’t they work?– Users are unable to behave as requiredUsers are unable to behave as required

How many accounts with passwords do you have?How many accounts with passwords do you have? How many actual passwords do you have?How many actual passwords do you have?

– Users are unwilling to behave as requiredUsers are unwilling to behave as required Do you create strong passwords all the time?Do you create strong passwords all the time?

User motivationsUser motivations

Users underestimate their risk and the negative Users underestimate their risk and the negative outcomesoutcomes– Has anyone ever had a password compromised or misused?Has anyone ever had a password compromised or misused?– how concerned are you about shoulder surfing for your how concerned are you about shoulder surfing for your

passwords?passwords?– What could happen if someone could get into your email? What could happen if someone could get into your email?

Your blog? Your bank account?Your blog? Your bank account? Users are not held accountableUsers are not held accountable

– Who makes sure you don’t write down your password? Who Who makes sure you don’t write down your password? Who makes sure you don’t reuse passwords?makes sure you don’t reuse passwords?

Conflicts with social norms & self imageConflicts with social norms & self image– Have you ever shared a password with a friend/colleague?Have you ever shared a password with a friend/colleague?– Why wouldn’t you share your bank password with your Why wouldn’t you share your bank password with your

spouse?spouse?

Question: The real-world equivalent of good security is Question: The real-world equivalent of good security is locking your home or car to protect your belongings. Yet locking your home or car to protect your belongings. Yet those who follow good cybersecurity practices are those who follow good cybersecurity practices are perceives as “anal” or “paranoid.” Why the difference?perceives as “anal” or “paranoid.” Why the difference?

The processThe process

The methods for creating the product.The methods for creating the product.

In your organization:In your organization:– Who creates security policies and technologies for Who creates security policies and technologies for

the employees?the employees?– Who creates the security policies and technologies Who creates the security policies and technologies

for the customers/users?for the customers/users?

AEGISAEGIS– What are the benefits of this method?What are the benefits of this method?– What are its drawbacks?What are its drawbacks?– Do the methods change if your users are non-Do the methods change if your users are non-

technical?technical?

The panoramaThe panorama

The context of the products, the larger environmentThe context of the products, the larger environment

EducationEducation– Teaching concepts and skillsTeaching concepts and skills

TrainingTraining– Correct usage of security mechanisms through drills, monitoring, Correct usage of security mechanisms through drills, monitoring,

feedback, reinforcementfeedback, reinforcement– Should encompass all staff, not only those with immediate Should encompass all staff, not only those with immediate

access to systems deemed at riskaccess to systems deemed at risk AttitudesAttitudes

– Role modelsRole models

What training/education have you had on good passwords?What training/education have you had on good passwords? What training/education has your [favorite-non-technical-What training/education has your [favorite-non-technical-

person] had?person] had?– What do you think they should have?What do you think they should have?– How could that be provided to them?How could that be provided to them?

Tog’s adviceTog’s advice

Achieving balanceAchieving balance– User context and bad guy contextUser context and bad guy context– User task and authenticationUser task and authentication– Security and privacySecurity and privacy

““RingWall” metaphorRingWall” metaphor– Castlekeep, ramparts, town wall, outsideCastlekeep, ramparts, town wall, outside– Is this a reasonable metaphor?Is this a reasonable metaphor?

Question: Much of security and privacy concerns has Question: Much of security and privacy concerns has more to do with where your more to do with where your informationinformation is, than where is, than where you are. Does Tog’s same desire for flexibility of you are. Does Tog’s same desire for flexibility of privacy settings based on the user’s environment privacy settings based on the user’s environment apply? Do the same metaphors apply?apply? Do the same metaphors apply?