Research-Methods-Handbook.pdf

7/30/2019 Research-Methods-Handbook.pdf

1/37

Research Methods HandbookIntroductory guide to research methods or social research

Stuart MacDonald & Nicola Headlam, CLES

CLES


2/37

2 3

Centre or Local Economic StrategiesExpress Networks 1 George Leigh Street

Manchester M4 5DL

tel 0161 236 7036 fax 0161 236 1891

[email protected] www.cles.org.uk

CLES is a registered charity no.1089503Company limited by guarantee no. 4242937 vat no. 451 4033 86

CLES Consulting is the trading name of CLES European Research Network

Company limited by guarantee no. 2467769 vat no. 519 493 812

ISBN: 1870053656

Te Centre or Local Economic Strategies (CLES) is a not-or-protthink-tank, consultancy and network o subscribing organisations

specialising in regeneration, local economic development and localgovernance, which was ounded in 1986.

About CLES Preace

Eective research methods are the tools by which inormation is gath-ered. Without the appropriate design and use o research methods, we

are unlikely to gather quality inormation and as such create a shakyoundation to any review, evaluation or uture strategy. For CLES, thegroup o research methods contained within this handbook are the tool

box, and like any tools need to be used in the right way - or the rightjob. Research methods, i understood and used appropriately can makeyour job a lot easier. At CLES we use a range o research methods inour policy and consultancy work and are skilled in using them. How-

ever, we do not believe that the use o research methods is the preserveo so called experts and in all instances requires sophisticated knowl-edge and practice. Clearly, to be experts in their use, requires practice,but like any tool, the basic principle behind its use can be understood

and applied, by all. Furthermore, or those who have no intention oactually using a research method, it is important, perhaps in your workin overseeing or commissioning research activity, to know what the

purpose o particular research methods are. Tis enables you to assessthe appropriateness o their use.

Tis handbook, thereore attempts on the one hand to explain anddemystiy the world o research methods, whilst on the other it seeksto provide a starting point or their use. In this, we are not suggestingthat using research methods is easy, but merely that it can be appreciat-

ed and undertaken by practitioners and non-research experts. We hopeyou enjoy this handbook which is linked to our annual training pro-gramme. Tis handbook, reects our wider charitable aim to develop

and improve the perormance o the regeneration and local economicdevelopment sector. In this, we hope this handbook goes some way in

helping to address some o the persistent issues aced by local placesand communities and contributes to ensuring positive local change.

Neil McInroy

Chief Executive, Centre for Local Economic Strategies.

printed on100% post-consumer recycled paper


3/37

Introduction 7

ypes o method 8

Quantitative Research Methods 11Quantitative survey 11Secondary data collation and analysis 20Statistical analysis 24

Qualitative Research Methods 35

Social surveys/questionnaires 35Interviews 39Discussion Groups 43Workshops 47Observation 50

Visual echniques 53

Research Issues 59

Glossary 61

Contents


4/37

6 7

We are all users o social research. How we apprehend and absorbinormation and use our critical aculties eeds what we believe about

the social world.

From surveys reported in the media to market research, opinion poll-

ing and large scale datasets such as the British Crime Survey. SocialResearch Methods are the tools used to explain social phenomena andoten it is more possible to challenge conclusions i you are at least

conversant with the variety o methodologies and tools applied.

What is this book about?Tis book provides an introduction to the reader to a whole range oresearch methods. It aims to introduce a toolkit o methods, explain-

ing how to use them, their appropriateness and some o the pitalls ousing them in practice.

Te sections explain in turn:

What is the method?

When should it be used? What do I need to consider? What is the output? How should it be analysed?

Examples from practice Pros/Cons Further Reading

Tere is also a glossary o technical terms at the back o the book.

Who should use this book?Tis handbook will be useul or those working in local authorities, re-

generation partnerships or other public agencies where research is con-ducted and commissioned. It can help in undertaking research in theselection o appropriate methods, making decisions about the mix omethods, and the planning o a research strategy where it is necessary

to make explicit judgements about a hierarchy o evidence, the weightaorded to the various elements, and how they might combine to give arounded perspective. It can also help in understanding research propos-

als, and the methodologies presented to address a particular researchquestion.

Introduction


5/37

8 9

In any orm o research, you will be required to either count things and/or talk to people. We can broadly classiy research methods using this

distinction. Tese two types o research method and their output dataare classied as:

Quantitative - as the name suggests, is concerned with trying toquantiy things; it asks questions such as how long, how manyor the degree to which. Quantitative methods look to quantiy

data and generalise results rom a sample o the population ointerest. Tey may look to measure the incidence o various

views and opinions in a chosen sample or example or aggregateresults.

Qualitative concerned with a quality o inormation,qualitative methods attempt to gain an understanding o the

underlying reasons and motivations or actions and establishhow people interpret their experiences and the world aroundthem. Qualitative methods provide insights into the setting o a

problem, generating ideas and/or hypotheses.

Te ollowing table provides a breakdown o the key eatures o each o

these categorisation o research method and data.

ypes o method

Quantitative Qualitative

Aim The aim is to count things inan attempt to explain what isobserved.

The aim is a complete, detaileddescription o what is observed.

Purpose Generalisability, prediction, causalexplanations

Contextualisation, interpretation,understanding perspectives

Tools Researcher uses tools, such assurveys, to collect numerical data.

Researcher is the data gatheringinstrument.

Data collection Structured Unstructured

Output Data is in the orm o numbers andstatistics.

Data is in the orm o words, pictures orobjects.

Sample Usually a large number o casesrepresenting the populationo interest. Randomly selectedrespondents

Usually a small number o non-representative cases. Respondentsselected on their experience.

Objective/Subjective

Objective seeks precisemeasurement & analysis

Subjective - individuals interpretation oevents is important

Researcher role Researcher tends to remainobjectively separated rom thesubject matter.

Researcher tends to become subjectivelyimmersed in the subject matter.

Analysis Statistical Interpretive


6/37

10 11

Quantitative methods are research techniques that are used to gatherquantitative data, data that can be sorted, classied, measured. Tis

ollowing section outlines the core quantitative research methods usedin social research.

Quantitative survey

What is the method?Surveys are a popular method o collecting primary data. Te broad

area o survey research encompasses any measurement procedures thatinvolve asking questions o respondents. Tey are a exible tool, whichcan produce both qualitative and quantitative inormation dependingon how they are structured and analysed. In this section we ocus on

the quantitative use o surveys, and in later sections we explore themore qualitative use o survey methods.

When should it be used?

When you need to generate primary data rom a large number o

sources to answer your research question. Surveys are a useul a meanso gathering data rom businesses, community organisations and resi-dents, and survey research is one o the most important areas o meas-

urement in applied social research. However, health warnings need tobe attached to the use o quantitative surveys and careul considerationneeds to be taken beore embarking on any large-scale survey.

What do I need to consider?In undertaking a survey it is important to understand who you want tosurvey, how you are going to select them, how you are going to surveythem, what you want to ask them and how you are going to organisethe task. Te ollowing section outlines some key considerations that

need to be made beore embarking on a large-scale survey.

Population A number o questions about the proposed popula-tion or a survey need to be considered. Such as:

Quantitative Research Methods

1.Howmany

roadsmustam

anwalkdown

?

a)lessthan1

0

b)10to20

c)morethan

20

d)dontknow


7/37

12 13

Can the population be counted? Some populations willbe easy to count, in a given geographical area there will be

secondary data sources that will give you a population count(Census), in a membership organisation there may be a list o allmembers, however in a newly arrived ethnic community such as

the recent arrivals of Polish and Ea stern European communitiesthere is less chance that you can ob tain a reliable count o thepopulation. A bias in your survey results can occur i the surveysample does not accurately represent the population. Having a

count o the population is also important in order to establishthe signicance o your results to allow a generalisation to thepopulation as a whole.

Are there language issues? Respondents may have vary-ing capacities or being able to complete written sur veys or

questionnaires. While telephone and street surveys do notrequire the respondent to be able to read or wr ite in English,postal surveys involve respondents completing the survey orquestionnaire themselves. You should consider the oer o help

in sel-administered surveys or respondents to complete a ormeither in person or over the telephone, this will help addresspotential language or basic skills issues. I surveying an ethnicminority population you may wish to translate quest ionnaires

into community languages, or have people who speak the com-munities language to assist where necessary.

What are the geographic restrictions? Te geographic spreado the population to be surveyed will determine the methodused or collecting your data. I you are surveying people rom a

particular location or organisation it may be possible to conducta survey using an interviewer, however i you have a populationsample that is geographically dispersed then you would look touse a dierent method, such as a telephone or postal survey.

SamplingTe sample is the section o the wider population that will be engaged

in the survey and sampling is the process o identiying who you willaim to contact rom that population. Te word population is used todescribe the target group, and while this may be the national popula-

tion as a whole, it may also be a smaller group such as lone parents, or

business members o a Chambers o Commerce in a particular location.Detailed consideration o sampling needs to be made to ensure the

validity o your results, and the ollowing issues need consideration:

Who is the respondent? Te rst thing you need to under-stand is who your respondent is going to be. Tis is the personthat will provide the data you are asking or. I the survey isdistributed amongst households, who in particular will be lling

in the survey? Do you want to specify who the survey is to becompleted by? And do you understand why you are specifying

this person? e same is true when surveying organisations orgroups. A survey will have much greater success i it is directed

to the right respondent. Identiying the person best suited tocompleting a survey will help to increase the response rate andgenerate more accurate data.

What is your sampling rame? A sampling rame is a list omembers o a population rom which members o a sample are

then selected. A sampling rame needs to be accurate, complete,up-to-date and relevant to the purposes o the survey orwhich it is to be used. Once you have an established samplingrame, depending on its size you may need to adopt a sampling

technique to extract your nal sample. For example randomsampling, simple random sampling or stratied sampling (seeurther reading or more details on sampling techniques).

Are response rates likely to be a problem? With any survey,you need to look at the prole o t he people who did responded

and satisy yoursel that they are about the same as the people

who didnt respond and also, that theyre about the same asthe overall population that youre sampling. I you send out asurvey to a population, which is 50% male, and 50% emale, but

your responses are 80% rom emales then your ndings willnot represent your target population. Response rates can below or surveys, under 20% or a postal survey is not uncom-mon. However, all the considerations in this section can help to

improve your response rate.

Statistical signicance: Understanding your population,sample size, and response rates are important or calculating


8/37

14 15

interval and condence levels, which are vital in determininghow many people you need to interview in order to get results

that reect the target population as precisely as needed. Youcan use online calculators to establish this type o inormation,but it is important to understand the terms and the reasons or

doing this (see section on statistical analysis or more detail).

FormatIt is important to understand what ormat o survey you are looking toundertake. Tere are broadly two survey ormats that you may use and

it is important to understand which you are using:

Cross-sectional surveys are used to gather inormation ona population at a single point in time. An example o a cross-sectional survey would be a questionnaire that collects data on

peoples experiences o a particular initiative or event. A cross-sectional survey questionnaire might try to determine therelationship between two actors, like the impact o a pro-

gramme o activity on the level o benets claims or example.

Longitudinal surveys gather data over a period o time. Tiswould allow analysis o changes in the population over time andattempt to describe and/or explain them. Te three main typeso longitudinal surveys are trend studies, cohort studies, andpanel studies (or more details see urther reading). A longitu-

dinal study will also seek to determine the relationship betweenactors, but the dierence is that the examination will be o achange in actors over time, so or example the relationship

between health and employment.

QuestionsTere are a whole range o questions to be asked in survey design,such as: What types of questions can be asked? How complex will/canthe questions be? Will screening questions be needed? Can quest ionsequence be controlled? Will lengthy questions be asked? Will long

response scales be used? Here we outline the main types of questionsused in quantitative surveys:

Closed questions these have a number o possible answersin a list or respondents to choose rom (e.g. a closed questionabout the sources o unding or a community project would

ask respondents to choose rom a list o categories, such as NewDeal or Communities, Neighbourhood Renewal Funding and so

on). Usually, closed questions include an other option to enablerespondents to add any categories that have been omitted;

Ranking scales these are most commonly used when trying toascertain the level o importance o a number o items. A list ochoices are provided and respondents are asked to put them in

order (e.g. when undertaking a easibility study or a new towncentre, a question using a ranking scale may show a list o items

that are commonly ound in town centres and ask respondentsto rank which ones are most important to them);

Sliding scales these are used to discover respondents s trengtho eeling towards an issue. Respondents are given a series o

statements and asked how much they agree or disagree withthe statement by using a sliding scale where numbers representdierent strengths o eelings. For example, 1 = strongly agree

and 5 = strongly disagree.

Write questions that are clear, precise, and relatively shortBecause every question is measuring something, it is important oreach to be clear and precise. Your goal is or each respondent to inter-pret the meaning o each sur vey question in exactly the same way. Iyour respondents are not clear on what is b eing asked in a question,

their responses may result in data that cannot or should not be appliedin your survey ndings.

Do not use loaded or leading questionsA loaded or leading question biases the response given by the respond-ent. A loaded question is one that contains loaded words. Loaded or

leading questions may hint to the respondent how you expect thequestion answered, or example Do you think your neighbourhood isstill run down?, by including the word still a bias is introduced as itpresupposes that the respondent thought the area was previously run

down.

Ambiguous or compound questions can be conusing, leaving respond-

ents unsure as to how to answer. Compound questions are ones thatask several things which might require dierent answers, or exampleWould you like to see more community suppor t ofcers on the streets,


9/37

16 17

It is vitally important to conduct a trial run or pilot o any survey, asthose that have designed a survey and are close to its subject, may take

or granted that the questions and layout will work as a survey with thewider intended population. A survey may be piloted with colleagues orriends that have the same level o involvement in the subject you are

surveying as the wider intended population. Feedback should be soughton the ease upon which the sur vey can be ollowed and completed. A pi-lot survey may also be conducted with a subset o the selected sample.Tis would give opportunities to detect and resolve problems beore

they obscure or distort the result o the wider survey.

allowing a reduction in investment in CCTV?. e respondent maywish to provide multiple answers to this question, answering yes to

having more community support ofcers, but disagreeing with thereduction in investment or CCV. See the section on urther readingor more inormation on question types and constructing sur vey ques-

tions.

AdministrationTe costs, required acilities, time, and personnel needed to conduct aneective survey are oten underestimated. Te most common resource

underestimated is time. You need to actor in time to pilot or testyour survey, time to deliver your sur vey, time to give respondents to

complete surveys and then have them returned (this may be via mailand thereore take time to return), and you also need to actor in thetime required to analyse surveys. When conducting a large scale survey,

inputting data to generate your analysis can be ver y time consuming.Te best approach is to oten work up your timeline backwards romwhen you need your results, calculating the time required or each step,this way you can establish when things need to start by.

How should it be used?

Selecting the type o survey you are going to use is one o the mostcritical decisions in many social research contexts. In a similar way tointerviews, surveys can be delivered in a variety o ways:

postal surveys; telephone surveys;

email/internet surveys;

street surveys/administered surveys.Te delivery method or any sur vey should be careully considered, andin many ways will be decided by consideration o actors listed above,such as population, sample s ize and respondent. Having a good under-standing o these will inorm the best method o delivery. For example,

i the survey is to be distributed to a particular local authority ofcerrole across the country, then a postal or email survey would work best,as it is likely there will be over 350 in the population, geographically

dispersed and literate.

Pros Cons

Postal Can reach a large geographical area No clarication available duringcompletion.

People are used to completingpaper-and-pencil surveys

Need a motivated population toreturn the survey

Can take the survey with you andcomplete it anywhere and anytime

Respondents must be able to read,see, and write

Great or sensitive issues

Telephone/administered

Inormation is obtained immediately Possible bias rom theadministrator

Can explore answers withrespondents

Higher level o resources

email/internet Negligible distr ibut ion costs Respondent must be onl ine

Only acceptable answers can be

allowed (validation)

Respondents must be able to

use a computer, a mouse, and/orkeyboard

Require the question to beanswered

Respondent must be able to use aweb browser

Can give respondent links that giveadditional explanation

Reliant on technology that can ail


10/37

18 19

ExcelMicrosot Excel is useul or data summary, presentation, and or other

basic statistical analysis. Te program provides a set o data analysistools called the Analysis ToolPak which you can use to save steps whenyou develop complex statistical analyses. You provide the data and pa-

rameters or each analysis and the tool uses the appropriate statisticalmacro unctions and then displays the results in an output table. Sometools generate charts in addition to output t ables. e Analysis ToolPakis not loaded by deault, instructions or installing it, along with guides

on how to use it can be ound on the Microsot website.SPSS (Statistical Package or Social Scientists)SPSS is among the most widely used program for statistical analysis insocial science. Tis is a dat a analysis package or quantitative research.It is particularly useul or the analysis o survey data as it covers a

broad range o statistical procedures. Tere are other packages availablesuch as SAS, Stata or Minitab however all are expensive to purchase,especially i only to be used or a one o survey. It may be possible towork with an academic institution to utilise their stat istical packages,

and organisations such as the Cathie Marsh Centre or Census and Sur-vey Research (CCSR) provide training on the use o these packages.

For more detail on data analysis see section on Statistical Analysis.

Further reading

A practical guide to sampling, National Audit Ofcehttp://www.nao.org.uk/publications/Samplingguide.pd

Question Bank is an inormation resource, in the eld o social re-search, with a particular emphasis on quantitative survey methods.http://qb.soc.surrey.ac.uk/

A general introduction to the design o questionnaires or sur-vey research, University o Leedshttp://www.leeds.ac.uk/iss/documentation/top/top2.pd

Te Centre or Applied Social Surveys, University o Southampton runs a programme o short courses in survey methods across the UK.http://www.s3ri.soton.ac.uk/cass/programme.php

What is the output?

Survey data is the question answers, such as yes or no or perhaps anumber, where a person has ranked a question on a scale. Te surveydata output will depend on the way in which the survey was construct-ed, it will be shaped by the survey questions asked, the ormat o the

survey itsel and the method in which data was collected. For example,i the survey was completed by the respondent, in a written orm, thenyou will have a collection o written documents which require analysis

o the question answers. I the survey has been completed by a re-

searcher, then a more sophisticated method o data collection may haveoccurred e.g. tallies and counts o responses. I using an internet oremail survey, a computer programme may have collected the data in a

ormat which can easily be analysed. Consideration o the output needsto be made at the outset o the process, and time considerations needto be given as to how this data will be collected and analysed.

How should it be analysed?

Beore you can input your data in a computer program or applicationyou will need to undertake a process o coding. Tis involves assign-ing a code (oten numeric) to each possible answer in your survey. Soi question 1 in your sur vey asked the gender o the respondent, you

may seek to code the answer male with the number 0, while you mayseek to code the answer emale with the number 1. Establishing thesecodes on the distributed questionnaire can help at data entry time, but

obviously has the downside o putting numbers on the questionnairethat are o no relevance to the respondent and thereore could makethe questionnaire look more conusing than it needs to.

Web based programmesInternet based survey tools can distribute your survey via email andalso collect your results, oten allowing you to view your results as they

are collected in real-time. You can download live graphs and chartso the responses, and oten lter the responses and dig down to getindividual responses. While this oers signicant benets there needs

to be careul consideration o the pros and cons o email or internetsurveys and whether this method o collection suits the population youare targeting.

SurveyMonkey

SurveyMonkey is anonline survey toolthat enables peopleo all experiencelevels to createtheir own surveysquickly and easily.It has an onlinesurvey designer,

which containsmany questions andormats. It collectsresponses andanalyses them inreal time, producingcharts and graphswith availableinormation. Allresponses can bedownloaded in avariety o ormatsto allow urtherstatistical analysis incomputer packagessuch as SPSS.

www.surveymonkey.com


11/37


12/37

22 23

Case study: Using secondary data to create a baseline

Baseline assessments reer to a number o headline indicators or statistics or a specicarea at a particular moment in time e.g. the percentage o unemployed economically ac-tive males in Newcastle-upon-Tyne in 2002. Baselines are particularly useul when meas-uring the impact o a regeneration project or programme, as by knowing what the area

was like beore it commenced, it enables an evaluator to gauge the extent the project/programme has changed an area.

Working with baseline data relating to local social, economic, cultural and environmental

conditions is a core eature in many policy interventions. In developing and updating abaseline, you may need to:

access secondary data (as detailed above) for establishing and up-dating baselines;

develop baselines retrospectively. is entails deciding key indica-tors and collecting data for a period of time in the past (e.g. thestart of a regeneration programme);

revise existing baseline indicators to ensure they reect local priori-ties and are SMART;

recommend new baseline indicators where gaps exist and devisingentirely new baseline assessments where one exist;

review and aligning baseline indicators with those used nationally(e.g. Quality of Life indicators, Best Value Performance Indicators,Public Service Agreement (PSA) indicators and National Floor Tar-get indicators) whilst maintaining a local focus;

identify the best sources of data, frequency of updates and leadresponsibilities or collection to aid uture baseline updates;

collect and collate primary (e.g. survey) data to inform the baselineassessment.

Further reading

How ofcial statistics are collected has improved a lot in recent years,

the neighbourhood statistics website is very user-riendly and usesmaps, graphs and tables or a wide range o ofcial statistics, present-ing data at many scales, rom the smallest unit o data collection; theSuper Output Area (population 1000-1500) to Parliamentary Constitu -

ency level, Local Authority level or regional Government Ofce Area.

http://www.neighbourhood.statistics.gov.uk/

Te Oce or National Statistics website is also excellent and con-tains all census and ofcial data as well as population projections and awealth o data on the economy.

http://www.ons.gov.uk

Oxord Consultants or Social Inclusion are experts in manipulat-ing datasets in order that they can help to inorm decision making.

Tey specialise in map data.

http://www.ocsi.org.uk/

Pros Cons

Robust, accurate data enabling comparison acrosstime/area

Health warnings around use o statistics need tomake sure o appropriateness in context

Very credible and clear picture can emerge whenpresented well

Using only national statistics can suggest overlyclear picture reality is oten messier.

As part o mixed method strategy can providebedrock or eld o enquiry and suggest otherappropriate techniques/interventions

You have no direct control over what is collected,only how it is presented and manipulated.


13/37

24 25

Statistical analysis

What is the method?

Statistical analysis is a mathematical method o interrogating data.

Tis is done by looking or relationships between dierent sets o data.Statistical analysis can be complex, and this ollowing section aims toexplain some o the basic considerations, to an audience without an as-

sumed mathematical background. At the end o this section there are awide variety o links to ur ther reading, which can help you through theprocess o statistical analysis.

Tere are two types o statistics:

Descriptive statistics: numerical summaries of samples (whatwas observed);

Inferential statistics: from samples of populations (what couldhave been or will be observed).

It is important to understand which type o statistics you are workingwith beore embarking on analysis.


Te general idea o statistical analysis is to summarise and analyse data

so that it is useul and can inorm decision-making. You would analysedescriptive statistics i you wanted to summarise some data into ashorter orm, where as, you would use inerential statistical analysiswhen you were trying to understand a relationship and either general-

ise or predict based on this understanding. Statistical analysis, througha range o statistical tests , can give us a way to quantiy the condencewe can have in our inerences or conclusions.

Statistical analysis should only be used where there is a clear under-standing o the reasons or doing so. Te use o statistical tests (as

detailed above) will provide you with valuable ndings i you know howto interpret the results and use them to inorm your research.

What do I need to consider?

VariablesA variable is any measured characteristic or attribute that diers ordierent subjects. Quantitative variables are measured on an ordinal,interval, or ratio scale, whereas qualitative variables are measured on

a nominal scale (note in SPSS the Interval and Ratio levels are groupedtogether and called scale). Tere are a range o variables that need to beunderstood, dependent/independent, controlled/continuous/discrete

in the application o statistical tests. Te independent variable answers

the question What do I change?, the dependent variable answers thequestion What do I observe? and the controlled variable answers thequestion What do I keep the same?. A variable which can have any

numerical value is called a continuous variable (e.g. time). A variablewhich can only have whole numbers (integers) is called a discrete var i-able (e.g. the number o people in a group). It is important to under-

stand the variable you have or analysis o data in statistical packagessuch as SPSS.

InerenceI working with inerential statistics you need a sound understandingo your population (the set o individuals, items, or data, also calleduniverse) and your sample (a subset o elements taken rom a popula-

tion). See the section on quantitative surveys or urther discussion onpopulations and samples. We make inerences (conclusions) about apopulation rom a sample taken rom it, thereore it is important thatpopulation and sampling is well understood, as any error will inuence

your inerences (conclusions). In some situations we can examine theentire population, then there is no inerence rom a sample.

Condence & Signicance Te condence interval is an interval estimate o a popula-

tion parameter, this is the plus-or-minus gure reported in, or

example, newspaper or television opinion poll results. I youuse a condence interval o 4 or ex ample, and 54% percent oyour sample picks one answer, you can be sure that i you hadasked the question o the entire relevant population, between

50% and 58% would have picked that answer (plus or minus 4).Tere are three actors that determine the size o the condence


14/37

26 27

interval or a given condence level. Tese are: sample size,percentage and population size (see below).

Te condence level tells you how sure you can be that thisinerence is correct. Most social science researchers use the 95%

condence level, which means you can be 95% certain; whilethe 99% condence level means you can be 99% certain. W henyou apply the condence level and the condence intervaltogether, you could say that you are 95% sure that between 50%and 58% would have picked that answer.

In statistics, a result is called statistically signicant i it is unlikely tohave occurred by chance. In statistics, signicant means probablytrue, and not import ant. Te ndings o your research may be provedto be true but this does not necessarily mean that the ndings are

important. In social science, results with a 95% condence level areaccepted as signicant.

Factors that afect the condence interval

Te condence interval is aected by three actors. Tese are the sam-

ple size, percentage and population size.

Sample SizeTe larger your sample, the more condent you can be that theiranswers truly reect the population. Te relationship between thecondence interval and sample size is not linear. An example can be

ound below:

Survey 1 Survey 2

Sample 1,000 2,000

Population 20,000 20,000

% o respondents answeringyes to a specic question

50% 50%

Condence Interval +/-3.02 +/-2.08

PercentageTe condence interval is also determined by the percentage o

the sample that provides the same answer. Te condence interval

increases the closer the percentage is to 50%. In survey 1 (above) thecondence interval or a value o 50% is 3.02. Tis condence interval

would all to 0.6 i the survey returned a value o 99% or 1%.

It is important that the survey sample size is considered or statistics

where 50% o the population answer both yes and no as this is whenthe condence level is broadest and so provides the general level o ac-curacy or a sample.

Population Size

Te population size reers to the number o people within a group thathave a similar characteristic. Tis could be the total number o peopleliving in a town or the number o people with a more specic attributesuch as suering rom a disability or residents rom a specic ethnicgroup. Population size is of greatest importance when the population is

relatively small and is known.

Examples

Condence A survey o 1,000 households has been completed, in a town o 20,000households. 54% of households felt that crime had the largest impact on their quality oflie. Using a 95% condence level a condence interval o 3.01 can be assumed. So you

can say that between 51% and 57% o the townss population eel the crime has the larg-est impact on quality of life.

Signicance A survey is distributed to all 20,000 households in a town, there are 1,000responses to the survey, equal to a 5% response. In accepting an interval level of 3, thesample size needed or signicant results at the 95% condence level is 1013, thereore

the response rate is just short o signicance at the 95% level.

Te signicance o change over time in s urvey ndingsIn measuring the condence interval o survey data when survey

results are compared over time, it is important to understand i, orexample, economic activity has changed over time or i the change inresults is caused by survey error. o understand whether actual changehas taken place, this requires the condence interval o t he dierence

between the two means to be tested (see urther reading or a l ink to aweb tool or measuring the condence interval between two means).


15/37

28 29

Example

Survey 1 nds that economic activity stands at 49% using a sample o 1,000 residents.Another sample is selected one year later. Survey 2 nds that 51% o residents are eco-nomically active. In this case the 95% condence interval is rom -0.05 to 0.03 meaningthat we cannot be sure whether the economic activity rates have actually increased or

whether this is a result of survey error. is is because the 95% condence interval hasvalues which are either side o zero.

I economic activity increases to 55%, the 95% condence interval is rom -0.09 to -0.01

meaning we can be 95% condent that economic activity has actually increased.

Considerations: Both surveys must be based on a sample that is representative o thepopulation. e sample used in survey 2 also needs to be independent from the sampleused in survey 1.

Cross-tabulationCross-tabluation is about taking two variables and tabulating the

results o one variable against the other variable. Tis can be done quitesimply in data analysis tools such as Microsoft Excel or SPSS. A cross-tabulation gives you a basic picture o how two variables inter-relate,

so or example you may have a question in your survey about employ-ment, by running a cross tabulation o the survey data obtained or thisquestion against that o age or gender or example (or both), would g iveyou a table showing the employment status o both males and emales,

broken down by the age ranges you coded in your survey. Tis can pro-vide quite powerul levels o inormation and is a useul way o test ingthe relationships between variables.

Statistical testsFor more complex statistical analysis there are a range o statisticaltests that can be applied to your data. o select the right test, you need

to ask yoursel two questions:

1. What kind of data have you collected?2. What variables are you looking to establish a relationship

between?Choosing the right test to compare measurements can be a tricky one,

as you must choose between two amilies o tests: parametric and non-parametric:

Parametric tests include Mean, Standard Deviation, t test,analysis of variance (ANOVA), Pearson correlation, regression

(linear and non linear);

Non-parametric tests include Median, interquartile range,Spearman correlation, Wilcoxon test, Mann-Whitney test,Kruskal-Wallis test, Friedman test.

Choosing the right testChoosing between these two amilies o tests can be difcult. Te

ollowing section outlines some o the basic r ules or deciding whichamily o tests suits your data.

You should choose a parametric test if your dat a is sampled from

a population that ollows a normal distribution (or Gaussiandistribution). Te normal distribution is a pattern or the distri-bution o a set o data, which ollows a bell shaped curve. Tis

means that the data has less o a tendency to produce unusuallyextreme values, compared to some other distributions.

You should choose a non-parametric test if the populationclearly does not ollow a normal distribution. Where valuesmay be o the scale, that is, too high or too low to measure,a non-parametric test can assign values too low or too high to

measure.

What do these tests tell you?

Parametric testsMean - Te mean is more commonly called the average, however

this is incorrect i mean is taken in the specic sense oarithmetic mean as there are dierent types o averages: themean, median, and mode.

Standard Deviation - Te standard deviation measures thespread o the data about the mean value. It is useul in compar-ing sets o data, which may have the same mean but a dierent

range.

t test - Te t-test assesses whether the means o two groups arestatistically dierent rom each other. Tis analysis is appropri-ate whenever you want to compare the means o two groups.


16/37

30 31

Analysis o variance (ANOVA) Tis is used to test hypothesesabout dierences between two or more means as in the t-test,

however when there are more than two means, analysis ovariance can be used to test dierences or signicance withoutincreasing the error rate (ype I).

Pearson correlation Tis is a common measure o the cor-relation between two variables. A correlation o +1 means that

there is a perect positive linear relationship between variables.A correlation o -1 means that there is a perect negative linear

relationship between variables.

Regression (linear and non linear) - A technique used or themodelling and analysis o numerical data. Regression can beused or prediction (including orecasting o time-series data),

inerence, hypothesis testing, and modelling o causal relation-ships.

Non-parametric testsMedian - Te median is the middle o a distribution: hal the

scores are above the median and hal are below the median. Te

median is less sensitive to extreme scores than the mean andthis makes it a better measure than the mean or highly skeweddistributions. Te median income is usually more inormativethan the mean income or example.

Interquartile range - Te interquartile range (IQR) is thedistance between the 75th percentile and the 25th percentile.

Te IQR is essentially the range o the middle 50% o the data.Because it uses the middle 50%, the IQR is not aected byoutliers or extreme values.

Spearman correlation - Spearmans Rank Correlation is a tech-nique used to test the direction and strength o the relationship

between two variables. In other words, its a device to showwhether any one set o numbers has an eect on another set onumbers.

Wilcoxon test - Te Wilcoxon test compares two paired groupso data. It calculates the dierences between each set o pairs,and analyses the list o dierences.

Mann-Whitney test - Te Mann-Whitney test is a non-paramet-ric test or assessing whether two samples o observations come

rom the same distribution, testing the null hypothesis that theprobability o an observation rom one population exceeds theprobability o an observation in a second population.

Kruskal-Wallis test - A non-parametric method or testingequality o population medians among groups, using a one-way

analysis o variance by ranks.

Friedman test - Te Friedman test is a nonparametric test thatcompares three or more paired groups.

What is the output?

Te output o statistical analysis will depend on the statistical test youapply to your data, a detailed understanding o the test is required to be

able to interpret the results. Te output will most probably be urthertables o data, with a number o things being reported. It is importantto understand the inormation you need rom a table o results, as youmay only require a single gure, but be presented with a range o inor-

mation which may be conusing i you are new to st atistical analysis.


Microsot ExcelMicrosot Excel includes a collection o statistical unctions, within the

add-on Data Analysis ToolPak. Excel can analyse descriptive statisticsat a simple level and when used eectively, can be very useul in theexploratory analysis o data, cross tabulations (pivot charts), viewing

data in graphs to detect errors, unusual values, trends and patterns andsummarising data with means and standard deviations. However, Excelis o very limited use in the ormal statistical analysis o data unlessyour experimental design is very simple. e Analysis ToolPak is also no

easier to use than more ormal statistical packages, however there areplenty o guides and tutorials to be ound on the internet.

Formal Statistical Packages (SPSS, SAS, Stata)Inerential statistics are more oten analysed in specialist statisticalpackages such as SPSS which provide greater functionality comparedto Excel. Te package used by the researcher oten depends on which


17/37

32 33

package the researcher is amiliar with and has access to. Tese ormalstatistical packages can summarise data (e.g. requencies), determine

whether there are signicant dierences between groups (e.g. t-tests,analysis o variance) and examine relationships among var iables (e.g.correlation, multiple regression). Further, these packages can produce

charts, graphs and tables rom the results o the analysis.

Pros Cons

Microsoft Excel Commonly used and widely available It is not possible to see a record othe analysis you have previouslyconducted

Easy to use or basic data analysis Statistical analysis is only possible idata is sorted or in blocks

Easy to import inormation rom otherpackages.

Limited by space MS Excel has asize limitation o 256 columns andover 65,500 rows meaning it haslimited capacity or analysing largerdatasets

Creating and amending charts is simple

FormalStatisticalPackages(SPSS, SAS,Stata)

Widely used. Expensive to purchase.

More recent versions are more userriendly than earlier versions (menus toselect rather than having to use syntax)

Need to buy add-ons to get ullunctionality

Allows a wider range o statistics test tobe conducted compared to Excel

Output isnt user riendly orbeginners

Easy to analyse survey / questionnaireresponses

Charts are poor quality and dicultto amend - need to copy inormationinto Excel

File size is only dependent on yourcomputers capacity

Survey data can be given assigned labels

It is easy to analyse sub groups o a largedataset

Further reading

Guide to Good Statistical Practice - Tis resource is based at the Sta-tistical Services Centre, University o Reading and consists o a serieso guides on good statistical practice, intended primarily or researchand support sta in development projects. Guides can be downloaded

in HTML and PDF on subjects such as Data Management and Analysis.Links include: training courses and workshops; consultancy; resources(such as publications, sotware, external links)

http://www.rdg.ac.uk/ssc/publications/guides.html

Introduction to Central endency, David Lanehttp://cnx.org/content/m10942/latest/ - A useul guide ex plainingsome basics to Statistical Analysis.

Statsot Electronic extbook,http://www.statsot.com/textbook/stathome.html - Tis ElectronicStatistics extbook oers training in the understanding and applicationo statistics.

Simple Interactive Statistical Analysis,http://home.clara.net/sisa/ - SISA allows you to do statistical analysisdirectly on the Internet. User riendly guides are available or statisticalprocedures.

Excel For Statistical Data Analysis,http://home.ubalt.edu/ntsbarsh/excel/excel.htm#rintro

Raynalds SPSS ools - http://www.spsstools.net/ - A website oeringtools and tips for users of SPSS software, the site oers an archive of400+ sample SPSS syntax, scripts and macros classied by purpose, as

well as an FAQ, tips, tutorials and a Newbies Corner. It invites contri-butions from other SPSS users to create a shared, open-source resource.

Choosing the correct Statistical est -http://www.ats.ucla.edu/SA/mult_pkg/whatstat/deault.htm

Condence interval between two means - Te ollowing link pro-vides a tool or measuring the condence interval between two meanshttp://psychlops.org/cgi-bin/R.cgi/bincon2.R


18/37

34 35

Qualitative methods are generally associated with t he evaluation osocial dimensions. Qualitative methods provide results that are usually

rich and detailed, oering ideas and concepts to inorm your research.Qualitative methods can tell you how people eel and what they think,but cannot tell you how many o the target population eel or think

that way as quantitative methods can.

Social survey/questionnaire

What is the method?Social surveys are a questionnaire-based method o research that canproduce both qualitative and quantitative inormation depending onhow they are structured and analysed. Tis section ocuses on the use

o surveys to collect and analyse qualitative data. Many o the issuesand considerations are the same as or the quantitative use o surveys,and more detail can be ound in the earlier section o this handbook.


Questionnaire surveys can be used in a wide range o settings and togather a variety o dierent types o inormation. You may be evaluat-ing a programme in which a wide range o projects have been com-missioned, and want to gather the views o a wide range o project

managers, or you may be measuring the impact o an initiative on thebusiness community in a specic geographical area. A small-scale quali-tative survey may b e conducted to explore in more detail the ndingso qualitative research.

What do I need to consider?Many o the considerations or a social survey are the same as or aquantitative survey, however we dene a social survey as one whereless statistical rigour is required, where sample sizes are not as large,

and with results not expected to be signicant o the wider population.A social survey may have a greater ocus on collecting rich and detailedqualitative data.

PopulationA number o questions about the proposed population or a social

survey need to be considered. Such as are there language issues? And

Qualitative Research Methods

1.Isthistherealli

fe?

yes/no

Ifno,pleasean

swerquestion2

2.Isitjustfan

tasy?

yes/no

Pleaseprovider

easonsforyour

answer

.........................

.........................

..........................

........................

.........................

.........................

....................

..........................

....

..............................................

....


19/37

36 37

what are the geographic restrictions? ese are the same issues as forquantitative surveys.

SamplingTe sample is the section o the wider population that will be engaged in

the survey. Detailed consideration o sampling still needs to be made evenwhen not striving or statistical signicance. It is still important to under-stand who the respondent is and what your sampling rame is going to be.

Format

A social survey will usually be a cross- sectional survey used to gatherinormation on a small sample population at a single point in time.An example o a cross-sectional survey would be a questionnaire thatcollects data on peoples experiences o a particular initiative. How-ever, a qualitative survey could equally be used in a longitudinal st udy,

perhaps returning to particular individuals over time to measure theimpact o an intervention on the direction o someones lie.

QuestionsTere are a whole range o questions to be asked in relation to surveydesign, such as: What types of questions can be asked? How complex

will the questions be? Will screening questions be needed? Can ques-tion sequence be controlled? Will lengthy questions be asked? Will longresponse scales be used? A social survey will be more interested in qual-itative ndings, in recording peoples opinions and perceptions, and

thereore will make more use o open questions where respondents cangive their own responses to a set question. Open questions will beginwith what, why, how, or describe, to elicit rich qualitative inormation.

Open questions can be used in a variety o ways:

Usage Example

As a ollow-on rom closed questions, to develop amore detailed response.

I answering yes to question 7, please provide thereasons or this

To nd out more about a person, their thoughts,needs, problems, etc.

Why is that so important to you?

To get people to realise the extent o theirproblems.

What eect does this have on your amily lie?

To get people to refect on the impact osomething or some change.

How has this made a dierence to you?

AdministrationTe costs, required acilities, time, and personnel needed to conduct an

eective survey are oten underestimated, even when it is not on a largescale. Tere should be an administrative system in place to deal with thequestionnaires or when they are returned/completed. Tis may include

numbering the questionnaires, recording what action has been takenwith them, entering the results into a spreadsheet/database etc.


Surveys can be carried out by phone, post, email, website or ace-to-ace, or detailed pros and cons o these deliver y methods see the ear-lier section on qualitative surveys. In collecting rich qualitative surveydata, the most eective method would be via ace to ace, administeredsurveys, as the researcher would be able to use prompts to encourage

people to give more detailed answers. Tis does however introduce abias, which needs to be understood and controlled as much as possible,i.e. by using standard prompts. In qualitative surveys, it is necessarythat the interviewer conduct the interview with total objectivity, so

that respondents are not inuenced by any outside source in theirresponses. For this reason, interv iews should be conducted by well-trained and qualied interviewers.

What is the output?

Te data that a social survey can produce is very much dependent onhow the questionnaire is constructed. However, the data can be veryuseul or providing an overall picture o the way in which a project orprogramme is being implemented and how eectively it is impacting

upon its target audience. Qualitative data output will be in a text, audioor picture ormat, and each answer may be very dierent rom another.Tis can make collection o data more difcult, and a way o collating

data needs to be considered early in the process.


Te Quantication o Qualitative Survey DataSurveys can be analysed by collating the requency o responses to each

o the questions on the survey orm. Tis can be done manually using a


20/37

38 39

requency table, which can be easily set up on an Excel spreadsheet toanalyse descriptive statistics.

QSR NUD*IS and NVIVO are qualitative data analysis packages, whichenable non-statistical inormation rom interviews, group work, obser-

vations, audio, video, pictures or documents to be analysed accordingto chosen criteria. For example, it is possible to use the package to pullout all material relating to key words or phrases (e.g. neighbourhood

renewal) and then sub-divide the data into more specic areas o analy-sis (e.g. statement o use, problems, projects). Tis is a powerul piece

o sotware that can provide clarity to wide range o oten complicatedwritten or media materials.

Case study: Using surveys to evaluate a project

A programme targeted on helping young people back into work through training wants

to evaluate how well it is achieving its objectives. It uses a sur vey to canvas the viewsof young people who have been on the programme to date. e survey asks them closedquestions about what training they have attended and how useful they have found thetraining (on a scale of 1:4). e survey also uses open questions to ask young peopleabout what their plans are for the future as a result of the training (i.e. has it helpedthem to consider applying for full time work? Or further education opportunities?). equalitative data is analysed and this shows that the young people have gained in con-dence, are looking to go into ur ther education or training or have already secured jobinterviews in a range o occupational elds, however there is a distinct ocus on work inthe eld o construction.

e results of the survey are analysed and this provides conclusions about overall success

o the programme, which allows the programme manager to draw conclusions and con-sider design issues or making the programme more efective in the uture.

Further Reading

See Question Bank or details o question design -http://qb.soc.surrey.ac.uk/

Computer Assisted Qualitative Data Analysis (CAQDA) http://caqdas.soc.surrey.ac.uk/ - provides practical support, trainingand inormation in the use o a range o sotware programs designed

to assist qualitative data analysis. Also provides various platorms ordebate concerning the methodological and epistemological issues aris-

ing rom the use o such sotware packages.Research Observatory, University o the West o E ngland -http://ro.uwe.ac.uk/RenderPages/RenderHomePage.aspx - the site isdivided into topic areas with each topic area containing a number o

learning units and a collection o resources about a particular subjectrelated to research.

Interviews

What is the method?

One o the most popular and requently used methods o gatheringinormation rom people about anything is by interv iewing them. It isalso the most popular method used within the social sciences. Tere isa continuum o ormality around interviewing and it covers a multitude

o techniques, rom inormal chats maybe arranged as vox-popsright through to highly structured, ormal interviews, taped and tran-scribed.

Te dierent types and styles o interv iew elicit very dierent types oinormation. Conducting interviews is an interpersonal process and

as an investigator you must be very aware o your own behaviours andassumptions in the context. Interviews are not neutral social spacesand you must be respectul and maintain appropriate boundaries at alltimes.


Interviews are a qualitative method o research oten used to obtainthe interviewees perceptions and attitudes to the issues. Te key issue


21/37

40 41

with interviewing is making decisions about who are the key people totalk to and what type o interview are you going to use.

Interview StyleTere are three clearly identiable styles o interview- structured, semi-

structured and unstructured:

Structured - Follows a set o specic questions, which areworked through systematically. Tis type o interview is usedwhen the researcher wishes to acquire inormation where the

responses are directly comparable.Semi-structured - Tis is a more commonly used interview

technique that ollows a ramework in order to address key

themes rather than specic questions. At the same time it al-lows a certain degree o exibility or the researcher to respondto the answers o the interviewee and thereore develop thethemes and issues as they arise.

Unstructured - Tis method o interview does not ollowany predetermined pattern o questions or themes. Rather,the interviewer will address the issues as they emerge in theinterview. Te method is useul when the researcher wishes toexplore the ull breadth o a topic.

Interview ypeTese are some o the types o interviews:

Fact nder - Tis type o interview is used to obtain specicinormation rom an interviewee and usually includes struc-

tured or standardised interview questions (the wording o thequestions and the order in which they are asked is the same). Itis used when some inormation is already known and there is

a need to gain a more in-depth insight. An example o when aact nder interview would be appropriate is when inter view-ing a project ofcer as part o an evaluation o their project.Quantitative (or hard) inormation is usually already known

(such as outputs and unding data), thereore the interviewcould be used to discover qualitative inormation that the harddata cannot portray, such as the soter outcomes o the project.

Idea generator - In many respects, this type o interview is theopposite o the act nder interview. It is used when the inter-

viewer has no preconceptions about what might be discoveredover the course o the interview and results can be used to setthe parameters or ramework or the study. Interview questions

are loosely structured allowing maximum exibility to explorea range o issues. Idea generator interviews are usually appliedat the start o a research project in order to discover and exploreissues rom a particular group or community. For example, in

order to develop a community cohesion strategy, idea generatorinterviews may be used to nd out what community cohesionmeans to dierent groups in the community.

Exploratory - Tese are the most requently used type o inter-view as they are relevant to most types o research project. Tey

are usually conducted with representatives that have a strateg icrole to play in the research. Tese types o interview requiresome degree o prior knowledge about the research subject asthey are about testing hypotheses, making connections between

other elements o the research, ensuring the strategic t andprogressing the ndings o the research orward (e.g. seniorofcials rom a local authority may be interviewed using thismethod in order to nd out uture plans and priorities and how

they t in with others plans and priorities).

Experiential - Tis type o interview aims to draw out peopleseelings, perceptions and experiences over a specic periodo time (e.g. the duration o a regeneration programme orproject). Tis provides rich, in-depth material about how the

subject under investigation has aected an individuals lie ona personal level. Experiential interviews may be used to elicitinormation rom people who have beneted rom a communityproject or who live in an area that has received regeneration

monies. Tereore these interviews can map the eelings andimpressions that any changes have made and add a story to thequantitative or hard data.


22/37

42 43

Pros Cons

Face to Face People can be very generous withtheir time and expertise

Appropriateness o setting

Interpersonal dynamics andestablishing trust may yield insights

Balance responsibility to yourinterviewees and needs oinvestigation

Telephone In-depth examination o topicpossible

Can be time intensive

Can do more without travel-time,rom your desk

Less opportunity to establish rapport

When should I use this method?

Interviews are typically used when seeking the views and opinions opeople with a specic perspective. Tey can be conducted by phone

or ace to ace. Tey oer particular advantages in terms o acquiringinormation, which might not otherwise be shared in a group setting.

What type o data is produced?

Te nature o the data will vary depending on the specic type o inter-

view undertaken by the researcher. Some people preer to take theirown notes, others preer to tape and transcribe verbatim, a lot dependson the preerence o the interviewer.

How can I analyse and use the data?

Te inormation obtained rom interviews can be used in two key ways:

Tematic generation - identiying and drawing upon commonthemes across the interviews;

Citation - directly quoting parts o the interview in the mainbody o the report.

Quotes have to be reerenced properly. For example, you may wish to

reer to the title o the interviewee in identiying who made the quote(eg project manager). Remember that some inormation provided dur-ing interviews may be condential. In such cases , you should only reer

to the broad theme or argument being made rather than identiyingwho said it.

Further reading

op tips or interviewing Research Observatory, University o theWest o Englandhttp://ro.uwe.ac.uk/RenderPages/RenderLearningObject.aspx?Context

=7&Area=1&Room=3&Constellation=25&LearningObject=124

Discussion groups

What is the method?

Discussion groups (also known as ocus groups) are an example o atechnique imported into social research rom market research. Teyhave been widely used in political circles to road test policies. A dis-

cussion groups consists o a number o individuals you invite to discusstheir views on a part icular topic, typically involving between 6 and 12people, which is conducted specically to get a group o peoples viewson a subject. Groups can be constructed in order to attempt to recreate

demographics.


Discussion groups are best applied when r ich, in-depth material rom

a number o people is required. B eing part o a group oten createsa more relaxed atmosphere than a one-to-one interview. Tereore,inormation gathered rom discussion groups is oten more varied t hani participants had been interviewed on a one-to-one basis. Another

advantage o using discussion groups, as opposed to one-to-oneinterviews, is that they provide in-depth inormation rom a numbero individuals simultaneously, making it a time eective method ogathering data.


23/37

44 45


Practical issuesDiscussion groups usually last one hour or so and include between sixto twelve participants. Participant recruitment is very important andcan be done through a range o methods, including client contact lists,

existing networks and databases, advertising in appropriate public plac-es and via the media, and hanging around places asking people to joinin. Tese varied methods o recruitment mean that discussion groups

can be targeted at dierent participants, including groups traditionally

considered hard to reach, such as young people and people rom Blackand Minority Ethnic (BME) backgrounds. Incentivising participants ortheir time requires some ethical consideration. However, expenses or

travel should always be provided, as well as ood and sometimes vouch-ers or cash payment.

Decide on the make up o the groups - identiy the key groupsand individuals that you need to speak to depending on whatyou need to nd out. Identiy the individuals you need to

contact. You may know o people directly or you may require theassistance o others to provide you with a route in to ndingparticipants (eg project sta, community champions). I this isthe case, simply outline what you intend to do and enquire as

to who they think the best people to involve in the ocus groupwould be. However, be aware that you should tr y to attract arange o participants with dierent views and experiences andthat relying on one person to nd all o your participants may

limit this taking place.

Arrange a location - ideally the discussion group should be heldin a location that is amiliar to the participants, as this amiliar-ity reduces the anxiety o the participant. Te next step is tocontact all potential participants to invite them to the group,

tell them what it is about, and inorm them o the time, locationetc. Ensure that you have more contact names than you needor the discussion group as some people may not wish to beinvolved.

QuestionsTe types o questions that could be asked during a discussion group

can be similar to interview questions, such as act-nding, idea gener-

ating, exploratory and experiential. Te main rationale or choosing toundertake group discussions as opposed to interviews should not be

the type o questions you are asking, rather to whom you are askingthe questions. Within group discussions having things to show or todemonstrate can really help the discussion as people interact with each

other and the stimulus provided.

FacilitationFocus group acilitation is a very specic skill, groups are notoriouslysusceptible to dynamics and can be quite difcult to control, conse-

quently i your requirements or parameters are ver y tightly denedthen a ocus group may be inappropriate. Some people nd such situ-

ations intimidating and do not contribute as much as they would in aone-to-one situation whilst some people may aect the dynamics bydominating proceedings.

In preparing for the discussion, it is worthwhile having ashortlist o questions, ideas and thoughts on the topic. Te list

could be useul in starting the discussion and ensuring it owscontinuously.

Ask relevant and open questions so that the discussion hasbreadth. It is important that the group has a discussion ratherthan a question and answer session. Tereore try to steer clearo questions that are narrow a nd can be answered easily without

discussion.

Encourage group interaction and participation. All members of

the group should make a contribution to the discussion. ry toavoid just one or two people dominating.

Pursue, capture and develop emerging issues. A good facilitatorshould spot issues that are emerging in the discussion and tryto get the group to discuss them in more detail.

Try to ensure that the discussion remains focused on the keythemes or issues.

Recording the discussionTis can be done either through the use o a tape recorder or by ta k-

ing notes. ape recording the discussion is useul in ensuring that noimportant points are missed and enables the acilitator to ocus on


24/37

46 47

guiding the discussion rather than taking notes. However, you will needto make sure that you have a good quality tape recorder in order to

pick-up the group discussion. A good alternative is to have a note-takersit in on the discussion.


Discussion groups are used when seeking the views, perceptions and

opinions o people in an open orum. Tey are oten used when morein-depth inormation is required than that which can be gained rom a

questionnaire. Compared with interviews, they can be used when con-dentiality is not an issue and where it is elt that participants a re more

likely to contribute within a group setting rather than on a one-to-onebasis. Tey can oten be used to explore issues emerging rom othertypes o research (eg interviews, surveys) in more depth.

What is the output?

Te discussion group produces qualitative data about thoughts, views,experiences etc.


Use the inormation rom interviews to identiy the relevant themes

that emerge rom the discussions to put into your evaluation report.Tere are also statistical packages that you can use to analyse this typeo data including:

NUD*IS: a qualitative data analysis package which enables non statis-tical inormation rom interviews, group work, observation etc to be

analysed according to chosen criteria. For example, it is possible to usethe package to pull out all material relating to key words or phrases (egneighbourhood renewal). I recorded, you may not need to transcribethe whole discussion but just make relevant notes rom the tape. Tis

will enable you to quote directly rom the discussion within your evalu-ation report, remembering to adhere to any issues o condentiality.

Pros Cons

You select and recruit group members Lack o interest in group could make recruitmentdicult

You can control the topic Participants do not have much to say or someparticipants dominate discussion

Interaction between participants may promptnew insights

May be unsuitable or researching sensitive issues

Further readingModerating ocus groups Te national centre or social researchprovides training courses in moderating or acilitating discussiongroups - http://www.natcen.ac.uk

Facilitating Workshops - Practical Tips for Workshop Facilitators,Seeds or Changehttp://www.seedsorchange.org.uk/ree/acilwsh.pd

Workshops

What is the method?

Workshops are a group-based method o research in which there is anemphasis on activity-based, interactive working. Te ocus is on every-one participating and undertaking the work. Tereore, when using this

type o research technique, the researcher acts as a acilitator, rather

than leading the discussion or activity.


Tere are a variety o reasons why it would be appropriate to hold work-

shop sessions, including:

Raising awareness (e.g. about a new unding stream and how toapply);

Capturing views and inormation (e.g. about local serviceprovision);


25/37

48 49

Outcome ocusedIt is imperative that workshops have clear objectives and are grounded

in the required outcomes o the session. Key to the achievement o thisare the pre-workshop activities that are underta ken to design, plan andprepare or the workshop itsel (see above). Te emphasis on outcomes

is important or all concerned it enables a gathering o inormation,perceptions and responses to contribute to the overall research, whilstenabling participants to understand the ocus o the session, which,in turn, allows them to play a ull role. I your participants understand

your aims or the workshop, then the session is likely to be more pro-ductive.

What is the output?

Te output o a workshop will be dependant on the types o activities

undertaken, but may include ip chart mater ial, drawings and diagramsand lists o actors, possibly ranked. It is important that all materialsand notes rom the workshop are collated, analysed and ed into theresearch ndings.


QSR Internationals NUD*IS & NVIVO computer packages enablenon-statistical inormation rom group work, observations, audio,video, pictures or documents to be analysed according to chosen crite-

ria. Tese are powerul pieces o sotware that can provide clarity to awide range o oten complicated written or media materials (see sectionon qualitative survey analysis).

Further ReadingComputer Assisted Qualitative Data Analysis (CAQDA) http://caqdas.soc.surrey.ac.uk/ - provides practical support, training and inormationin the use o a range o sotware programs designed to assist qualitativedata analysis. Also provides various platorms or debate concerning

the methodological and epistemological issues arising rom the use osuch sotware packages.

Facilitating Workshops Practical Tips for Workshop Facilitators,Seeds or Changehttp://www.seedsorchange.org.uk/ree/acilwsh.pd

Building consensus (e.g. to take orward a drat strategy or ac-tion plan);

Developing skills and capacity (e.g. on how to implementemerging government policy).


PlanningWorkshops need to be well planned, this will oten involve establishingthe date/time/location o the workshop as early as possible; inviting

potential participants to the workshop by letter/ema il and requestingconrmation o their attendance; distributing background papers andthe objectives/required outcomes o the workshop in advance; and pre-

paring practical aids or use in the workshop itsel (e.g. photos, maps,ipcharts, presentations, models).

Workshops vary in size according to the nature o the sub ject, thespecic group involved and the required outcomes o the session.Workshops can contain as ew as 4 participants a nd as many as 25.Te length o the workshop will var y depending on actors such as

the planned activities, the time available a nd the required outcomes.Workshops can range in duration rom one hour to ull day sessions.However, it is important to be aware o the time pressures under which

people work and to ensure that the scheduling and duration o theworkshop(s) is appropriate.

InteractiveTe emphasis during workshops is on par ticipation. Tis can be encour-aged through stimulating debate (e.g. posing questions) a nd encourag-ing collaborative working (e.g. group activities). A variety o mecha-

nisms can be employed to encourage interaction, including:

Brainstorming;

Model making; Physical and mental mapping; Ranking and prioritisation;

Drawing and photography; Role play.

Te techniques selected need to be tailored according to the specicgroup o participants (e.g. strategic decision makers, project sta,

young people) and the outcomes required.


26/37

50 51

Observation

What is observation?

Observation, sometimes reerred to as participant observation or

ethnography is the key method o anthropology and in itsel canconsist o a mix o techniques; inormal interviews, direct observation,participation in the lie o the group, collective discussions, analyses o

personal documents produced within the group, sel-analysis, and lie-histories, notes, diaries and transcripts are oten kept and the observa-tion method can generate a lot o wr itten material which the investiga-tor must synthesize.

Participant observation is usually underta ken over an extended periodo time, ranging rom several months to many years. An extended

research time period means that the researcher will be able to obtainmore detailed and accurate inormation about the people he/she isstudying.


Observation is more appropriate when seeking to uncover:

Observable detailsLike daily time allotment. For example, the popular managementconsultancy technique o the time and motion study is a version oobservation. Te investigator watches the activities and actions o

people involved in a process and works out the specic time allocationdevoted to every single step, with the objective o improving efciencyby cutting out unnecessary or time consuming steps.

Group dynamicsI the subject o your enquiry is a collective, in this context more likelyto be a partnership board or steering group rather than a tribe or

sub-culture, then close attention to the dynamics o the interactionbetween the people involved can be very illuminating. Te obser vationmethod highlights interpersonal relationships and the investigator canreect upon social proximity and distance, observe relationships and

explore body lanuage and other behaviours.

More hidden detailsLike taboo behaviour. Observation can be eective in ex ploring or

exposing secrets or the underlying realities o situations, researcherscan discover discrepancies between what participants say and otenbelieve should happen (the ormal system) and what actually does

happen, or between dierent aspects o the ormal system; in contrast,a one-time survey o peoples answers to a set o questions might bequite consistent, but is less likely to show conicts between dierentaspects o the social system or between conscious representations and

behavior.


Observation as part o a mixed methods approachObservation may be appropriate as a part o your research strategy but

it is unlikely that it would stand alone in the research contexts thatwe have been describing. It is also worth remembering that it reliesvery heavily upon the judgements, assumptions and prior knowledgeand experience o the observer themselves.

Reliability vs ValidityParticipant observation (whether overt or covert) is not the most reli-

able research method. Such studies, by their very nature, are impos-sible to repeat and reliability can be ur ther questioned in terms othe extent to which the presence o the obser ver actually changes the

behaviour o those being studied. As soon as you do or say anythingat all, you have slipped rom the role o observer to participant, thisboundary can be very hard to maintain.

Participant observers study people in their natural environment, gain-ing a depth o insight into behaviour that comes not simply rom close,detailed, observation but also rom the researchers own experiences

within the group being studied a technique that provides rst handinsights into why people behave as they do. Participant observationdoes not prejudge issues and events (in t he way a questionnaire may,

or example) and, or these reasons it is possible to argue that such amethod provides data that has a high level o validity.

Skills requiredParticipant observation requires a great deal of skill and commitmentrom the researcher. Te success or ailure o the research will hinge


27/37

52 53

on such actors as the a bility to t-in with the people being studiedand the ability to communicate with groups members on their level

and terms. It will also, at dierent times, require tact, clear and careulobservation, the ability to separate the role o participant rom thato observer and so orth. In other words, beore committing yoursel

to participant observation you need to be certain you have the time,resources and skills required to carry this type o research.

What is produced?

A key eature o participant observation is that data should be col-lected in ways that cause as little disruption as possible to the ordinary

activities o the research context. Te recording o inormation largelydepends on the research situation. Fieldnotes are generally kept andsometimes it is poss ible to use tape recorders and video recorders.

Whichever methods o recording inormation are used it is importantto be detailed and to devise a system that allows easy retrieval o inor-mation.

How should the data be analysed?

Analysis and interpretation o data is undertaken in a similar way toanalysing and interpreting data gathered by other qualitative researchmethods, as detailed in other sections o this handbook.

Pros Cons

Observation Deep and nuanced picture canemerge

Relies on observer to read socialreality accurately

A fexible method that can reactto events / ideas, ollow leads,pursue avenues o research thathad not been considered

Hard to maintain observer role

Gives a researcher insights intoindividual and group behaviourand it may allow them toormulate hypotheses thatexplain such behaviour

Can need a signicant timeperiod

Further reading

Collecting data through observation, Web Centre or Social Re-search Methods -http://www.socialresearchmethods.net/tutorial/Brown/lauratp.htm - Understand the advantages and disadvantages o observational

research compared to other research methods.

Visual techniques

What is the method?

Visual techniques in social research oer an interesting, stimulatingand interactive approach to gathering inormation. Tey are appropri-

ate in a variety o situations, as they ull numerous unctions. Visualmethods such as drawing, painting, video, photography and hyperme-dia oer increasingly accessible and popular resources or research.

ypes o visual research that you might want to consider:

Cartoon test presented with a picture o a car toon depicting aspecic situation, the consultees are then asked what they thinkthe character would do, say or think in response to another

character.

Completion technique using the cartoon test above, the char-acter is shown thinking or saying something but the sentenceis left uncompleted. Participants are given the oppor tunity to

make suggestions to complete the sentence.Collage/concept boards uses a range o images that can be

used to represent or describe the subject or discussion (e.g.services, project, issues etc). In this way, the participants can

identiy the subject with a range o eelings and moods. Tereare two ways to approach this technique; either the par ticipantsrespond to prepared boards or they construct their own collage

or concept boards.


28/37

54 55

Ideas board this board invites participants to jot down theirideas on post-it notes and add them to the board which is

grouped by theme.

Mind mapping visually representing inormation in aninteresting ormat without the limits or ormality o standardwritten text. Te open owing ormat appears to support thenatural thinking process, which is thought to go on randomly

and in a nonlinear way.

Money well an interesting way o asking participants toprioritise future actions or developments. Participants are givena certain amount o ake money, which they can place on theoptions displayed.

Grati or ideas wall a strip o paper is hung on the wall ac-companied by shapes, such as speech bubbles. Te participantsare able to write comments a bout a research topic or discussion

statement.

Photographic research consultees are either provided with aseries o photographs by the researchers or given a disposablecamera or mobile phone and asked to take their own. Dependingon the nature o the research project, these photos might in-

clude depictions o the local physical environment and/or reecthow consultees view their community, including what they likeand dislike about it. Te use o photographs in research can beused to evoke eelings or trigger memories that lead to perspec-

tives and explanations that would not have b een unlocked usinga more conventional research technique.

Film and video - video cameras are particularly well suited asdata gathering technologies or ethnography, participant obser-vation, oral history, lie history, etc, preserv ing things that are

not preserved in even the best researchers eld notes. Similarly,tape recordings preserve audible data not available in even t hemost careully annotated transcripts.


Deciding to conduct visual research will be dependant upon the type

and scale o the research that you are undertaking. It may be appropri-

ate to conduct this type o work at the start o the research process as away o highlighting issues to be exa mined urther during the course o

the process, or it may be something that is developed as part o a blendo evaluation methodologies. Visual techniques can be used in manysettings, as an alternative to more traditional methods and may be

particularly useul as:

A method for eectively engaging hard to reach groups within

areas (e.g. young people). Pictures and photographs can helpevoke opinions and allow the use o imagination in expanding

on a scene.

Oering an alternative to traditional discussion groups, yet stillbeing able to draw out the rich va riety o qualitative inorma-tion rom participants.

For use within workshops, providing a task based activity to getmembers o a group working and thinking together.

A method of producing tangible outcomes at the end of the

research process (e.g. series o community generated impactsillustrating how local people view the local area).


Consider who should be involved in this type of visual researchand that the technique

Research-Methods-Handbook.pdf

Documents