What are Online Research Methods? Tristram Hooley & Jane Wellens

What are Online Research Methods?

Tristram Hooley & Jane Wellens

What are online research methods?• Research methods that utilise the internet as

medium for research

• Sometimes referred to as internet mediated research

• Includes research methods designed to investigate both online & offline phenomenon

Types of ORM• online questionnaires• synchronous and asynchronous interviews• virtual ethnographies• online experiments• web analytics & content analysis• + others

Online Research Methods - Challenges

• Draw on a wide range of methodological, theoretical and disciplinary traditions

• Understanding the range of tools, environments and online cultures is a precursor to successful research

• Technological and social environments change year on year

• Increased blurring of the online and offline environments creates a need for new composite methodologies

• Potentially poses new and different ethical considerations

Advantages of online research?• Huge uptake of ORM, especially online questionnaires• Useful in particular situations e.g. researching online

populations and previously difficult to contact groups• Can mitigate distance and space and so

internationalise research• Can save time and money• Decrease personal risk• Different dynamics of communication:

- more neutral venue?- more thoughtful responses?- different participants open

up/contribute?

But….disadvantages• But issues of digital access and digital literacy can limit

who we talk to• Drop off rates for questionnaires high• Issues of identity verification• Challenges to building rapport in interviews• Circumstances of interview beyond interviewers control• Technical obstacles• Still need degree of technical competence and

institutional support (although this is getting easier all the time)

Online Questionnaires - key questions

What are your

research questions/

aims?

Who are you talking

to?Sampling

RecruitmentNon-response

How are you talking to them?

Question typeSurvey length

Language

Advantages of online questionnaires

• Increasingly common approach (familiar to respondents)

• Speed and volume of data collection• Savings in costs (to researcher)• Flexible design• Data accuracy• Access to research populations• Anonymity

Online questionnaires - challenges• Sample bias• Measurement error (Sax et al, 2003)• Non-response bias• Length, response and dropout rates• Technical problems• Ethical issues• Recruitment• Identity verification• Response rates• Identifying appropriate tools & implementation

Design issues

Usability

Accessibility

Doability

Will your questionnaire be easy to use?Or will the interface, colours and question types frustrate people and put them off?

Will your questionnaire be accessible?Or will it crash on old computers, render weirdly on different operating systems and be impossible to use with a screen reader?

Will your questionnaire be doable?Or are your multi-media dreams running ahead of your technical ability and are your data arriving in an unusable format?

Improving response rates• Make contact before mailing the survey e.g. an introductory

letter • Provide information that builds trust e.g. names and photos of

the researchers• Engage gatekeepers and encourage them to endorse the survey• Think about how your brand (e.g. the university) will be

perceived by the subjects • Provide clear instructions on how to complete the questionnaire;• Limit the amount of personal information you request• Use simple questionnaire format and avoid complex or open-

ended questions• Design survey so it takes approximately 10 minutes to complete;• Do not include more than 15 questions• Send one or two follow up reminders• Emphasise confidentiality (if appropriate)

Key resource: Fan, W., & Yan, Z. (2010). Factors affecting response rates of the web survey. Computers in Human Behavior , 26 (2), 132-139.

Some Software Options• Google Forms http://docs.google.com FREE• Lime Survey http://www.limesurvey.org FREE• BOS http://www.survey.bris.ac.uk CHEAP• SurveyMonkey http://www.surveymonkey.com • Snap http://www.snapsurveys.com • Using the functionality of your website or VLE• Bespoke systems

So….The quantity of information that may be generated, and the speed at which responses can be collected, can result in pleasing piles of data- but we should be wary of being seduced by sheer quantity; data is only useful if it is representative of the larger population.Wakeford (2000, 33)

Online Interviews – key questions • Why use online interviews?• What type of interview (synchronous or

asynchronous)?• One-to-one or group? • Text based or using audio or video?• What software/service will you use? • How are you going to record, retrieve and

analyse data?

Online Interviews: advantages• Carry out interviews with a very geographically

dispersed population• Interview individuals or groups who are often difficult to

reach, such as the less physically mobile or the socially isolated or those living in dangerous places.

• Provide savings in costs to the researcher• Supply ready transcribed interview data, quickly,

providing fast and cheap alternatives to face-to-face interviews

• Reduce issues of interviewer effect as participants cannot 'see' each other

Online interviews - challenges• Trust and rapport building• Design of interview schedule• Keeping people on topic• Dealing with technical hitches• Maintaining momentum• Dealing with ‘silence’• Guaranteeing the ethical rights of respondents

including informed consent, confidentiality and privacy

Some software options• Adobe ConnectNow

http://www.adobe.com/acom/connectnow/ • TinyChat http://tinychat.com/ (probably need to

upgrade for anything serious)• GoogleTalk http://www.google.co.uk/talk/• Many VLEs also have chat and virtual

classroom tools that can be used for this kind of purpose.

So...• The data collected by virtual interviews can be rich and

valuable to the researcher, but the potential of on-line research should not be exaggerated: many of the issues and problems of conventional research methods still apply in the virtual venue

• Moreover, it is unlikely that online interviewing is going to replace face-to-face interviewing but rather it is another option in the methodological ‘toolkit’

Online research ethics• There is currently a big push to embed research ethics

more formally in the culture of the social sciences.

• Yet there is far less of a consensus about online research ethics.

• This asks the question “is there anything special about the online environment that requires new set of ethical guidelines?”

Key resource: Economic and Social Research Council (2005). Research Ethics Framework.

Ethical decision making and Internet research

• Ess & AoIR (2002) stresses “Ethical pluralism” and argues that “there is more than one ethical decision-making framework used to analyze and resolve those [internet research] conflicts.”

• Note: the world of chatrooms, MUDs and MOOs and USENET newsgroups, described in this report in 2002 is now radically transformed.

Key resource: Ess, C. and AoIR Ethics Working Committee (2002) Ethical decision-making and internet research http://www.aoir.org/reports/ethics.pdf

Where does it fit in?One of the problems of mapping existing research ethics onto the online world is the difficulty of deciding how to categorise the spaces that you are observing/interacting with.

Broadcast

Public Private

Published Informal

Writing Speech

Personal

Anonymous Identified

Key resource: Internet Research Ethics: Digital Library, Resources Centre, and Commons - http://internetresearchethics.org/

Ethical Challenges“The great variety of human inter/actions observable online and the clear need to study these inter/actions in interdisciplinary ways …This interdisciplinary approach to research leads, however, to a central ethical difficulty: the primary assumptions and guiding metaphors and analogies - and thus the resulting ethical codes - can vary sharply from discipline to discipline, especially as we shift from the social sciences.

Ess, C. and AoIR Ethics Working Committee (2002)

Careful and critical use ORMs• So ORMs are not a shortcut `easy option’• Many issues and problems of onsite research remain• Divide between onsite/online methods inappropriate• Must be used, carefully and critically and appropriately

in light of each specific research topic

Attempting to undertake online data collection is far easier than successfully accomplishing it. For those who chose to perform it, they must do so deliberately and cautiously.(Best and Krueger, 2004)

While online methodological frameworks are in constant flux, change is not necessarily always progressive: there is a need for online researchers to practice their 'craft' with reflexivity. (Madge and O’Connor, 2005)

Further readingExploring online research methodshttp://www.geog.le.ac.uk/orm/

Tristram’s ORM bibliography on CiteULikehttp://www.citeulike.org/user/pigironjoe/tag/online_research_methods

Tristram’s blog http://adventuresincareerdevelopment.posterous.com/ I talk about ORM sometimes and technology often on this blog.