Structured Surveys in Social Research. What are surveys? One method of collecting, organising analysing data One method of collecting, organising analysing.

Structured Structured Surveys in Social Surveys in Social

ResearchResearch

What are surveys?What are surveys? One method of collecting, organising analysing One method of collecting, organising analysing

datadata Data collectionData collection - - is structured and systematic is structured and systematic

Info. on same variables from multiple cases (people, Info. on same variables from multiple cases (people, groups, orgs.) groups, orgs.)

Both quantitative and qualitative dataBoth quantitative and qualitative data Data analysisData analysis - - looking for systematic variation looking for systematic variation

between variables and casesbetween variables and cases Variable by case matrix form – like experimental Variable by case matrix form – like experimental

methodmethod But more difficult to attribute differences to experimenter But more difficult to attribute differences to experimenter

interventionintervention Multiple techniques Questionnaires most common;Multiple techniques Questionnaires most common;

But also: structured and in-depth interviews, direct But also: structured and in-depth interviews, direct observation techniques etc.observation techniques etc.

Survey v Participatory Survey v Participatory methodsmethods

SurveySurvey ParticipatoryParticipatory

Stre

ng

ths

Stre

ng

ths

Statistical inferenceStatistical inference

Qualitative Qualitative &Quantitative&Quantitative

GeneralisabilityGeneralisability

Stakeholder Stakeholder empowerment?empowerment?

Rapid ‘quick & dirty’ Rapid ‘quick & dirty’ methodmethod

Potentially less Potentially less resources req. resources req.

Flexible and highly Flexible and highly adaptiveadaptive

Weakn

esse

sW

eakn

esse

s

Intrinsically Intrinsically manipulativemanipulative-Power to Power to practitionerspractitioners-Ideological Ideological persuasionpersuasion

Iterative rigidityIterative rigidity

Causal Causal connections?connections?

Simplicity results in Simplicity results in misuse?misuse?

Mostly qualitative Mostly qualitative methodsmethods

Reliability & Reliability & generalisabilitygeneralisability

Short term fashion?Short term fashion?

The Steps in a Survey The Steps in a Survey ProjectProject

Establish the goals of the project - What you Establish the goals of the project - What you want to learn want to learn

Determine your sample - Whom you will Determine your sample - Whom you will interview interview

Choose interviewing methodology - How Choose interviewing methodology - How you will interview you will interview

Create your questionnaire - What you will Create your questionnaire - What you will ask ask

Pre-test the questionnaire, if practical - Test Pre-test the questionnaire, if practical - Test the questions. the questions.

Conduct interviews and enter data - Ask the Conduct interviews and enter data - Ask the questions. questions.

Analyze the data - Produce the reports. Analyze the data - Produce the reports.

Descriptive and Explanatory Descriptive and Explanatory Research 1Research 1

Survey research has two Survey research has two fundamental questions:fundamental questions: What? – DescriptiveWhat? – Descriptive

i.e. Social structure, market research, i.e. Social structure, market research, opinion pollopinion poll

Why? – Explanatory Why? – Explanatory TheorisingTheorising Grounded / Grounded / ex post factoex post facto theory theory Efficient data collection and analysisEfficient data collection and analysis

Descriptive and Explanatory Descriptive and Explanatory Research 2Research 2

Descriptive research – focusing the Descriptive research – focusing the topictopic Time frame?Time frame? Geographical location?Geographical location? Comparison of sub-groups?Comparison of sub-groups?

Explanatory researchExplanatory research Either causesEither causes of change of change And / or And / or consequencesconsequences of change? (i.e. of change? (i.e.

PIM)PIM)

Units of Analysis

-Village / Household/ Individual

Farming Farming Systems Systems

and Units and Units of of

AnalysisAnalysis

Obs1 Obs2 Obs3 Obs4

Theory Construction & Theory Construction & TestingTesting

Theory

Empirical level

Obs1 Obs2 Obs3 Obs4Empirical level

Conceptual

abstract level

Theory Construction

Theory Testing

Survey Design 1Survey Design 1

Classic design – experimental v control Classic design – experimental v control groupgroup Availability of controls? Availability of controls? Repeated measures for same group?Repeated measures for same group?

Panel design – same group over timePanel design – same group over time Longitudinal surveyLongitudinal survey Problem of re-visiting all panel members?Problem of re-visiting all panel members?

Quasi-panel design – diff groups over Quasi-panel design – diff groups over timetime

Survey Design 2Survey Design 2

Retrospective panel or experimental Retrospective panel or experimental design design Based on recall – selective memory?Based on recall – selective memory?

Cross-sectional design – most commonCross-sectional design – most common Matched experimental and non-experimental Matched experimental and non-experimental

groupsgroups Comparison at one point in timeComparison at one point in time

One shot ‘case’ study – most primitiveOne shot ‘case’ study – most primitive One group at one point of timeOne group at one point of time Limited use for evaluating causal processesLimited use for evaluating causal processes

SamplingSampling Two main types of sampleTwo main types of sample 1. Non-probability samples1. Non-probability samples

Some cases have greater or unknown selection Some cases have greater or unknown selection chancechance

2. Probability samples to achieve 2. Probability samples to achieve representative sample of population to representative sample of population to permit generalisationpermit generalisation Each case has equal or known chance of Each case has equal or known chance of

selectionselection Random selection from sampling frameRandom selection from sampling frame Sampling error – standard errorSampling error – standard error

Probability Samples 1Probability Samples 1

Four types – choice depends onFour types – choice depends on Resources - Desired accuracy, survey Resources - Desired accuracy, survey

method and availability of a good method and availability of a good sampling frame. sampling frame.

1. Simple random sampling (SRS)1. Simple random sampling (SRS) Determine sample sizeDetermine sample size Random number table selectionRandom number table selection Requires good sample frameRequires good sample frame

2. Systematic sampling 2. Systematic sampling Like SRS but simpler – same limitationsLike SRS but simpler – same limitations Proportional sampling i.e. every fifth Proportional sampling i.e. every fifth

member of frame – periodicity?member of frame – periodicity?

Probability Samples 2Probability Samples 2

3. Stratified sampling3. Stratified sampling Select stratifying variable(s) i.e. wealthSelect stratifying variable(s) i.e. wealth Proportion in different groups same as Proportion in different groups same as

populationpopulation Requires Requires a prioria priori knowledge of knowledge of

stratifying variablestratifying variable 4. Multi-stage cluster sampling4. Multi-stage cluster sampling

Area based i.e. random selection of Area based i.e. random selection of districts, streets and finally householdsdistricts, streets and finally households

Sample Size?Sample Size?

Size depends on two key factors:Size depends on two key factors: Degree of accuracy required (max 5% SE?)Degree of accuracy required (max 5% SE?) Population variability re. key study variablesPopulation variability re. key study variables

For small samples, a small increase in For small samples, a small increase in sample size can result in a large increase in sample size can result in a large increase in accuracyaccuracy

Variability of most heterogeneous Variability of most heterogeneous variables?variables?

Requirements of statistical techniques?Requirements of statistical techniques? Allow for non-response – be aware of bias!Allow for non-response – be aware of bias!

Non Probability SamplesNon Probability Samples

Where no reliable sampling frame is Where no reliable sampling frame is availableavailable

Or where populations are highly dispersedOr where populations are highly dispersed Or where no requirement for Or where no requirement for

generalisation from sample to populationgeneralisation from sample to population Exploratory analysisExploratory analysis Hypotheses generation Hypotheses generation Testing questionnairesTesting questionnaires

Purposive, quota and availability samplingPurposive, quota and availability sampling

From Concepts to From Concepts to QuestionsQuestions

Descending the ladder of abstraction!Descending the ladder of abstraction! Clarify concepts – functional v substantiveClarify concepts – functional v substantive Develop good indicatorsDevelop good indicators

Measurable: No’s & dimensions (space and time)Measurable: No’s & dimensions (space and time) Reliability – Consistency in repeated useReliability – Consistency in repeated use Validity – Ability to measure the concept as Validity – Ability to measure the concept as

intendedintended Consistency of meaning to different people !Consistency of meaning to different people ! Pilot test or borrow from existing Pilot test or borrow from existing

questionnairesquestionnaires Clarify concepts and indicators with iterationClarify concepts and indicators with iteration

Explanatory variablesExplanatory variables Dependent variables – i.e. Dependent variables – i.e. farmers level of farmers level of

relevant knowledge of information and skills relevant knowledge of information and skills and experience. and experience.

Independent variables –Independent variables – i.e.i.e. farmers farmers learning & info. gathering methods, learning & info. gathering methods, socioeconomic status, ethnicity, etc. – factors socioeconomic status, ethnicity, etc. – factors which might explain their knowledge of the which might explain their knowledge of the dependent variablesdependent variables

Confounding variablesConfounding variables – Other factors – Other factors related to both dependent and independent related to both dependent and independent factors which may distort the results and have factors which may distort the results and have to be adjusted for. to be adjusted for.

Open or Closed Format Open or Closed Format QuestionsQuestions??

Open formatOpen format questionsquestions Allows exploration of the range of possible Allows exploration of the range of possible

themes arising from an issue themes arising from an issue Can be used even if a comprehensive range of Can be used even if a comprehensive range of

alternative choices cannot be compiled alternative choices cannot be compiled

Closed or forced choice-format questionsClosed or forced choice-format questions Easy and quick to fill in Easy and quick to fill in Minimise discrimination against the less Minimise discrimination against the less

literate or the less articulate literate or the less articulate Easy to code, record, and analyse results Easy to code, record, and analyse results

quantitatively quantitatively Easy to report results Easy to report results

Design and Response Design and Response RatesRates

Short simple questionnaires usually attract Short simple questionnaires usually attract higher response rates than long complex higher response rates than long complex boring ones. boring ones.

Adding variety in the types of questions asked Adding variety in the types of questions asked will helpwill help

Make and give incentivesMake and give incentives?? Ordering of questionsOrdering of questions

Go from general to particular. Go from general to particular. Go from easy to difficult. Go from easy to difficult. Go from factual to abstract. Go from factual to abstract. Start with closed format questions. Start with closed format questions. Start with questions relevant to the main subject. Start with questions relevant to the main subject. Do not start with personal questions.Do not start with personal questions.

Other considerationsOther considerations

Avoid leading and double barrelled questionsAvoid leading and double barrelled questions Keep language simple (short) and preciseKeep language simple (short) and precise

Frame of reference: who and over what time periodFrame of reference: who and over what time period Anticipate interpretational difficultiesAnticipate interpretational difficulties

By respondents and enumerators – pilotingBy respondents and enumerators – piloting Appropriate questions – cultural sensitivityAppropriate questions – cultural sensitivity

Direct and indirect questionsDirect and indirect questions Be aware of prestige bias – i.e. wealth rankingBe aware of prestige bias – i.e. wealth ranking

Cultural Cultural SensitivitSensitivit

y?y?

Administering the Administering the questionnairequestionnaire

Advantages of self administered Advantages of self administered questionnairesquestionnaires

Cheap and easy to administer. Cheap and easy to administer. Preserve confidentiality Preserve confidentiality Can be completed at respondent's convenienceCan be completed at respondent's convenience Can be administered in a standard mannerCan be administered in a standard manner

Advantages of interview administered Advantages of interview administered questionnairesquestionnaires

Allow participation by illiterate, less well Allow participation by illiterate, less well educated people.educated people.

Higher response rate?? Higher response rate?? Allow clarification of ambiguity through dialogueAllow clarification of ambiguity through dialogue

ExamplExampleses

from from Sri Sri

LankaLanka

Surveys used to measure Surveys used to measure fish yields from stocked fish yields from stocked

village reservoirsvillage reservoirs Second Phase: 5 reservoirs belonging to Second Phase: 5 reservoirs belonging to 4 villages stocked, 1 control village4 villages stocked, 1 control village

1. Direct observation1. Direct observation Collective fishing and staggered harvestingCollective fishing and staggered harvesting

2. Longitudinal panel survey2. Longitudinal panel survey 7 day recall of household fish consumption7 day recall of household fish consumption Fortnightly, 41 wealth stratified householdsFortnightly, 41 wealth stratified households

3. Retrospective panel survey 3. Retrospective panel survey Participatory impact monitoring (PIM)Participatory impact monitoring (PIM) Stakeholder perceptions and yield recallStakeholder perceptions and yield recall

1. Longitudinal Survey -Mean 1. Longitudinal Survey -Mean monthly per capita monthly per capita

consumption by sourceconsumption by source

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Month

Mea

n m

on

thly

co

nsu

mp

tio

n (

kg/

cap

ita)

Major irrigation system 1.34 1.26 1.08 0.62 0.73 0.81 0.38 0.34 0.78 0.90 1.13 0.91 1.03

Other village tank 0.09 0.08 0.10 0.11 0.06 0.13 0.27 0.42 0.29 0.23 0.18 0.18 0.24

Stocked village tank 0.03 0.19 0.07 0.67 0.36 0.24 1.22 0.67 0.11

Nov 00 Dec 00 Jan 01 Feb 01 Mar 01 Apr 01 May 01 Jun 01 Jul 01 Aug 01 Sep 01 Oct 01 Nov 01

Monthly Yield by Species Monthly Yield by Species & Tank& Tank

0

20

40

60

80

100

120

140

160

180

200

Apr Jun Jul Aug Sep Jun Jul Aug May Jun Aug Jun Jul Aug Sep Oct Jun Jul Aug Sep Oct

GBW LHG SER LUN KBW

Tank / Month

Yie

ld (

kg)

Tilapia Spp. Channa striata Trichogaster pectoralis Cyprinus carpio Anabas testudineus

Tank Phase

Area at 50% FSL Ha

Yield estimate (kg ha-1 yr-1)

1. Direct2. Household

/ survey3. PIM

GURt

1

12.95 23.1

IMKt 9.05 48.7

IMK SERt

1.7578.6

IMK SERt

2

111.1

GBWt 4.21 59 342 19.5

LHGt 6.78 15.4 49.4 12.4

MAD LUNt 1.9 100.7

695171

MAD KBWt 0.95 84.6 281.1

Data ManagementData Management

Data ManagementData Management Spreadsheets: Lotus, Excel etc.Spreadsheets: Lotus, Excel etc.

Regular ‘symmetrical’ data matricesRegular ‘symmetrical’ data matrices Arithmetic calculationsArithmetic calculations

Databases: Access, FoxPro, File-Maker, Databases: Access, FoxPro, File-Maker, OracleOracle Relational Design - 3D v 2D :- Relational Design - 3D v 2D :- Large, irregular and longitudinal surveysLarge, irregular and longitudinal surveys Data Accessibility and exploratory analysisData Accessibility and exploratory analysis

Non-numerical Databases i.e. Nudist, Non-numerical Databases i.e. Nudist, InVivoInVivo

Statistical packages i.e. Minitab, SPSSStatistical packages i.e. Minitab, SPSS

Spreadsheet ApproachSpreadsheet Approach

Hse. Hse. No.No.

HH HH HeadHead

NamNamee11

AgAgee11

SeSexx11

NameName22

AgeAge22

SexSex22

11 SunilSunil SunilSunil 3434 MM FatimFatimaa

2626 FF

22 DusaDusann

DusaDusann

5555 MM

33 IndirIndiraa

IndirIndiraa

2424 FF KumaKumarr

44 MM

44 RanjiRanjitt

RanjiRanjitt

4747 MM

Database Relational Database Relational DesignDesign

Hse Hse No.No.

NameName AgAgee

SexSex

11 SunilSunil 3434 MM

11 FatimFatimaa

2626 FF

22 DusaDusann

5555 MM

33 IndiraIndira 2424 FF

33 KumaKumarr

44 MM

44 RanjitRanjit 4747 MM

Hse Hse No.No.

HH HH HeadHead

11 SunilSunil

22 DusaDusann

33 IndirIndiraa

44 RanjiRanjitt

‘One to many’ relationships

Primary Key

Primary Key

Database Benefits 1Database Benefits 1

Ease of Data entryEase of Data entry Organised ‘compact’ tablesOrganised ‘compact’ tables Automated coding systemsAutomated coding systems Entry forms look like questionnaire Entry forms look like questionnaire

formatsformats Multiple Users - online (web pages) or Multiple Users - online (web pages) or

offline synchronisationoffline synchronisation Data entry error checksData entry error checks

Through relational design Through relational design Other built in validation stepsOther built in validation steps

Database Benefits 2Database Benefits 2

Data AnalysisData Analysis Demands Demands a prioria priori planning planning Exploratory Analysis before higher level Exploratory Analysis before higher level

statistical analysisstatistical analysis Data reduction - cross-tabulation (Pivot Data reduction - cross-tabulation (Pivot

Tables)Tables) Integrated with MS office suite and Integrated with MS office suite and

statistical packages - SPSS, Minitab…statistical packages - SPSS, Minitab…

LimitationsLimitations

Exponential learning curveExponential learning curve Dependence on technical backup support Dependence on technical backup support

Fallback strategiesFallback strategies Structured design Limits flexibility / Structured design Limits flexibility /

iteration possibilitiesiteration possibilities Empiricism v anthropological view of Empiricism v anthropological view of

human behaviour as complex sets of human behaviour as complex sets of meaningsmeanings