from analysis plan to data collection Helen Maguire acknowledgements Katharina Alpers, Yvan Hutin
Mar 27, 2015
from analysis plan to data collection
Helen Maguire
acknowledgements
Katharina Alpers, Yvan Hutin
its logical: data collection follows the analysis plan ..
Research question:? Risk factors for leptospirosis
Study objectives:Estimate association between water exposure and disease
Design/ indicator:Case controlOdds ratio
Analysis plan:Dummy table
Data elementsNeeded:? Water exposure? Sick
Data collection:Interview
Individual items:? Swam in water ? Sick
Consolidationof theinstrument
data quality
• reliability
• accuracy
Data quality
data quality
• reliability– reproducibility/repeatability/precision
– ability of a measurement to give the same result or similar result with repeated measurements of the same thing
– refers to stability or consistency of information
• accuracy• ability of a measurement to be correct on the
average
Data quality
reliability and accuracy
Data quality
ReliableAccurate
Not reliableAccurate
ReliableNot accurate
Not reliableNot accurate
reliability and accuracy
Data quality
some essentials
1. question-by-question guide
2. train staff who collect data
3. standardize the data collection procedure
4. control instruments or specimen collection
5. validate
Data collection
1 question by question guide(q-by-q)
• short document
• each question/data item – item by item
• guidance as to how the data should be collected
• derived variables explained
Data collection
2 train field workers
• select good, experienced field workers
• walk through q-by-q guide
• explain where data is
• clarify how to record it
• simulate interviews with team
Data collection
3 standardize data collection
• interviewers– work in teams
– resolve issues in the whole group
• instruments/specimens/samples– calibration, standardization, packaging, media,
transport
Data collection
4 control instruments
• team checks instrument/samples before leaving
• all take responsibility for the instrument:– names and signatures
• investigator checks instruments/samples as they come
Data collection
5 validate
how would you validate/verify?
example - triangulation to estimate the proportion of blood units screened for HIV
(internal validation)
• interview laboratory manager• ? what is the number of units screened
• observe practices of the laboratory technician– structured observation guide
• ? proportion of units tested
• review of registers - ? a sample– proforma
• ? number of tests ordered, performed
Instruments
data might include
• facts– individual characteristics
• height, age, income
– environment
• housing, family size
– behaviours, practices
• alcohol or tobacco consumption
• judgements – attitudes/opinions
• indicators of socio-economic status …
• blood test results
• environmental samples Instruments
list some ways to collect data
Instruments
• proformas– clinical records
– surveillance records
– registers
• questionnaires
• sampling /laboratory results
• other data – socio-economic status derived from postcode –via
linkages
– denominator data
– reference data
checking the instrument(s) against the analysis plan
• make sure you can collect what you need for each variable /indicator
• suppress unnecessary data collection or questions at interview– those that do not be used in the analysis
Production of the instrument
is a questionnaire good?why?
• list 5 advantages and 5 disadvantages
advantages of questionnaires
• can reach a large number of people
• relatively easy and economic
• relate directly to study question
• provide quantifiable answers
• relatively easy to analyse
possible disadvantage of questionnaire
• bias …?
• how might it be introduced at this stage?
• how would you avoid it?
• pilot - check for leading questions
how to reduce bias
• structured questionnaire
• ensure high response rate
• random choice of interview partners (next birthday)
• train interviewers
disadvantages of questionnaires• provide only limited insight into a
problem– the range of possible responses is limited – the question maybe misleading
Unclear question can lead to • misunderstanding • misinterpretation
• do not allow for mistakes– must be right from the beginning – missing data hard to chase
how to avoid ?
pilot testing of the questionnaire
• check that the questionnaire is:– clear
– understandable
– acceptable
• check flow and skip pattern
• check coding
• estimate time needed
Production of the instrument
how would you administer a questionnaire?
questionnaires
• internet/email/post self completion
• interviewer-administered– face to face
– telephone
what makes a well designed questionnaire?
what makes a well designed questionnaire?
• good appearance (easy for the eye)
• short and simple
• numbering / flow /sign-posting
/instructions/where to return and how
• relevant and logical
introduction
• covering letter/ interview introduction– Who are you / you work for
– Why are you investigating
– Where did you obtain the respondent’s name
– How and where can you be contacted
– Guarantee of confidentiality
– Length of interview (be honest)
⇒ Usefulness of study should be clear to all respondents
• Good morning ,
My name is Katharina Alpers .... , I work for ……..
You may have been already informed that a survey on risk
factors for being stung by a jellyfish will be done this week in
Mahon. This study has been approved by the Spanish national
ethical committee. Only anonymous data will be analysed. You
have been randomly selected to participate in this study. Your
participation is voluntary. The interview is about 10 minutes
long. Are you able to help us?
thanks so much .....
questions
do you like to go swimming and do you mind being
stung by jellyfish?
Yes No
what is the jellyfish situation? Good Bad
versus
how often did you see jellyfish during the last week?
Once Twice Three times or more Never Don´t know
did you see more than an average of 33 jellyfish/m2 salt water surface on more than 3 occasions that you went swimming in the morning last week?
Yes No
versus
have you seen jellyfish on more than 3 mornings last week?
Yes No Don´t know
main question formats
• closed format forced choiceYes Always No Sometimes Don’t know Never
• open format free textWhat did you do to avoid being stung by jellyfish? Please describe : __________________________________________________________________________________
• when would open questions be good ?
• what problems might there be with open questions?
advantages of open questions
• exploration possible – to generate hypotheses
• useful for exploring knowledge and attitudes• qualitative research
• focus groups
• trawling questionnaires
disadvantages of open questions
• interviewer bias
• time-consuming
• coding problems
• difficult to analyse
• difficult to compare groups
advantages of closed questions
• simple
• less discrimination against less verbally expressive people
• easy to code, record, analyse
• easy to compare
disadvantages of closed questions
• restricted number of possible answers
• possible loss of additional information
Compromise
if yes specify : __________”
which of the following beaches have you visited during your stay in Menorca?
Lazareto beach Yes No Don´t know
Cala’n Porter Yes No Don´t know Rafalet
Yes No Don´t know
Macarella Yes No Don´t
know
Sa Mesquida Yes No Don´t know
checklist
rating scale
how often did you see jellyfish during the past week?
Always Sometimes Seldom Never
Mornings
Lunchtime
Evenings
rating scalenumericalhow severe was your pain after you were stung?
(please circle) 1 2 3 4 5 6 7
Not painful at all Very painful
analoguehow severe is your pain (put the tick on the line)
0 10
Likert ScaleRensis Likert, 1903-1981Five (or more) ordered response levels
Jellyfish also have the right to swim in the Mediterranean sea
I strongly disagree
I disagree
I neither agree or disagree
I agree
I strongly agree
problems and pitfalls
• avoid questions that ask two things at once - you won’t know which part people are answering:
have you seen or been stung by jellyfish?
• ambiguity.....
do you swim a lot?
problems and pitfalls• avoid jargon/abbreviations/slang
should jellyfish sting victims receive PEP?
(post exposure prophylaxis)
• avoid not mutually exclusive options
What is your age ? 16-20 20-25 25-30 35-40
summary
a well designed questionnaire:
• helps you answer your research question
• minimises potential sources of bias
-> increases the validity of the replies
• is more likely be completed
questionnaire validation
• use or adapt existing questionnaires – validated
• new questionnaires– need to be tested (pilot)
conclusion
• don’t forget to thank the interviewed persons
• tell them when the results will be available and where
take home messages
• think instruments, data sources, not only questionnaire
• list your indicators
• prepare your variables ->indicators
• prepare dummy tables
• polish, polish and polish to ensure good data quality
• thanks for your attention